1. 24 Apr, 2021 28 commits
    • Masahiro Yamada's avatar
      kbuild: remove CONFIG_MODULE_COMPRESS · d4bbe942
      Masahiro Yamada authored
      CONFIG_MODULE_COMPRESS is only used to activate the choice for module
      compression algorithm. It will be simpler to make the choice always
      visible, and add CONFIG_MODULE_COMPRESS_NONE in the choice.
      
      This is more consistent with the "Kernel compression mode" and "Built-in
      initramfs compression mode" choices. CONFIG_KERNEL_UNCOMPRESSED and
      CONFIG_INITRAMFS_COMPRESSION_NONE are available to choose no compression.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Reviewed-by: default avatarNick Desaulniers <ndesaulniers@google.com>
      d4bbe942
    • Masahiro Yamada's avatar
      kbuild: merge scripts/Makefile.modsign to scripts/Makefile.modinst · 961ab4a3
      Masahiro Yamada authored
      scripts/Makefile.modsign is a subset of scripts/Makefile.modinst,
      and duplicates the code. Let's merge them.
      
      By the way, you do not need to run 'make modules_sign' explicitly
      because modules are signed as a part of 'make modules_install' when
      CONFIG_MODULE_SIG_ALL=y. If CONFIG_MODULE_SIG_ALL=n, mod_sign_cmd is
      set to 'true', so 'make modules_sign' is not functional.
      
      In my understanding, the reason of still keeping this is to handle
      corner cases like commit 64178cb6 ("builddeb: fix stripped module
      signatures if CONFIG_DEBUG_INFO and CONFIG_MODULE_SIG_ALL are set").
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      961ab4a3
    • Masahiro Yamada's avatar
      kbuild: move module strip/compression code into scripts/Makefile.modinst · 65ce9c38
      Masahiro Yamada authored
      Both mod_strip_cmd and mod_compress_cmd are only used in
      scripts/Makefile.modinst, hence there is no good reason to define them
      in the top Makefile. Move the relevant code to scripts/Makefile.modinst.
      
      Also, show separate log messages for each of install, strip, sign, and
      compress.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      65ce9c38
    • Masahiro Yamada's avatar
      kbuild: refactor scripts/Makefile.modinst · ccae4cfa
      Masahiro Yamada authored
      scripts/Makefile.modinst is ugly and weird in multiple ways; it
      specifies real files $(modules) as phony, makes directory manipulation
      needlessly too complicated.
      
      Clean up the Makefile code, and show the full path of installed modules
      in the log.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      ccae4cfa
    • Masahiro Yamada's avatar
      kbuild: rename extmod-prefix to extmod_prefix · 7f69180b
      Masahiro Yamada authored
      This seems to be useful in sub-make as well. As a preparation of
      exporting it, rename extmod-prefix to extmod_prefix because exported
      variables cannot contain hyphens.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Reviewed-by: default avatarNick Desaulniers <ndesaulniers@google.com>
      7f69180b
    • Masahiro Yamada's avatar
      kbuild: check module name conflict for external modules as well · 1a998be6
      Masahiro Yamada authored
      If there are multiple modules with the same name in the same external
      module tree, there is ambiguity about which one will be loaded, and
      very likely something odd is happening.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      1a998be6
    • Masahiro Yamada's avatar
      kbuild: show the target directory for depmod log · 3ac42b21
      Masahiro Yamada authored
      It is clearer to show the directory which depmod will work on.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      3ac42b21
    • Masahiro Yamada's avatar
      kbuild: unify modules(_install) for in-tree and external modules · 3e3005df
      Masahiro Yamada authored
      If you attempt to build or install modules ('make modules(_install)'
      with CONFIG_MODULES disabled, you will get a clear error message, but
      nothing for external module builds.
      
      Factor out the modules and modules_install rules into the common part,
      so you will get the same error message when you try to build external
      modules with CONFIG_MODULES=n.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      3e3005df
    • Masahiro Yamada's avatar
      kbuild: remove unneeded mkdir for external modules_install · 4b97ec0e
      Masahiro Yamada authored
      scripts/Makefile.modinst creates directories as needed.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      4b97ec0e
    • Bhaskar Chowdhury's avatar
      scripts: modpost.c: Fix a few typos · f3945833
      Bhaskar Chowdhury authored
      s/agorithm/algorithm/
      s/criterias/criteria/
      s/targetting/targeting/   ....two different places.
      Signed-off-by: default avatarBhaskar Chowdhury <unixbhaskar@gmail.com>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      f3945833
    • Masahiro Yamada's avatar
      kbuild: fix false-positive modpost warning when all symbols are trimmed · 4475dff5
      Masahiro Yamada authored
      Nathan reports that the mips defconfig emits the following warning:
      
        WARNING: modpost: Symbol info of vmlinux is missing. Unresolved symbol check will be entirely skipped.
      
      This false-positive happens when CONFIG_TRIM_UNUSED_KSYMS is enabled,
      but no CONFIG option is set to 'm'.
      
      Commit a0590473 ("nfs: fix PNFS_FLEXFILE_LAYOUT Kconfig default")
      turned the last 'm' into 'y' for the mips defconfig, and uncovered
      this issue.
      
      In this case, the module feature itself is enabled, but we have no
      module to build. As a result, CONFIG_TRIM_UNUSED_KSYMS drops all the
      instances of EXPORT_SYMBOL. Then, modpost wrongly assumes vmlinux is
      missing because vmlinux.symvers is empty. (As another false-positive
      case, you can create a module that does not use any symbol of vmlinux).
      
      The current behavior is to entirely suppress the unresolved symbol
      warnings when vmlinux is missing just because there are too many.
      I found the origin of this code in the historical git tree. [1]
      
      If this is a matter of noisiness, I think modpost can display the
      first 10 warnings, and the number of suppressed warnings at the end.
      
      You will get a bit noisier logs when you run 'make modules' without
      vmlinux, but such warnings are better to show because you never know
      the resulting modules are actually loadable or not.
      
      This commit changes the following:
      
       - If any of input *.symver files is missing, pass -w option to let
         the module build keep going with warnings instead of errors.
      
       - If there are too many (10+) unresolved symbol warnings, show only
         the first 10, and also the number of suppressed warnings.
      
      [1]: https://git.kernel.org/pub/scm/linux/kernel/git/history/history.git/commit/?id=1cc0e0529569bf6a94f6d49770aa6d4b599d2c46Reported-by: default avatarNathan Chancellor <nathan@kernel.org>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      4475dff5
    • Masahiro Yamada's avatar
      kbuild: do not set -w for vmlinux.o modpost · 5ab70ff4
      Masahiro Yamada authored
      The -w option is meaningless for the first pass of modpost (vmlinux.o).
      
      We know there are unresolved symbols in vmlinux.o, hence we skip
      check_exports() and other checks when mod->is_vmlinux is set.
      
      See the following part in the for-loop.
      
          if (mod->is_vmlinux || mod->from_dump)
                  continue;
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      5ab70ff4
    • Masahiro Yamada's avatar
      kbuild: generate Module.symvers only when vmlinux exists · 69bc8d38
      Masahiro Yamada authored
      The external module build shows the following warning if Module.symvers
      is missing in the kernel tree.
      
        WARNING: Symbol version dump "Module.symvers" is missing.
                 Modules may not have dependencies or modversions.
      
      I think this is an important heads-up because the resulting modules may
      not work as expected. This happens when you did not build the entire
      kernel tree, for example, you might have prepared the minimal setups
      for external modules by 'make defconfig && make modules_preapre'.
      
      A problem is that 'make modules' creates Module.symvers even without
      vmlinux. In this case, that warning is suppressed since Module.symvers
      already exists in spite of its incomplete content.
      
      The incomplete (i.e. invalid) Module.symvers should not be created.
      
      This commit changes the second pass of modpost to dump symbols into
      modules-only.symvers. The final Module.symvers is created by
      concatenating vmlinux.symvers and modules-only.symvers if both exist.
      
      Module.symvers is supposed to collect symbols from both vmlinux and
      modules. It might be a bit confusing, and I am not quite sure if it
      is an official interface, but presumably it is difficult to rename it
      because some tools (e.g. kmod) parse it.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      69bc8d38
    • Masahiro Yamada's avatar
      arm64: move --fix-cortex-a53-843419 linker test to Kconfig · 987fdfec
      Masahiro Yamada authored
      Since commit 805b2e1d ("kbuild: include Makefile.compiler only when
      compiler is needed"), "make ARCH=arm64 (modules_)install" shows a false
      positive warning.
      
      Move the ld-option test to Kconfig, so that the result can be stored in
      the .config file, avoiding multiple-time evaluations in the build and
      installation time.
      Reported-by: default avatarNathan Chancellor <nathan@kernel.org>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Acked-by: default avatarWill Deacon <will@kernel.org>
      Tested-by: default avatarNathan Chancellor <nathan@kernel.org>
      Reviewed-by: default avatarNathan Chancellor <nathan@kernel.org>
      987fdfec
    • Masahiro Yamada's avatar
      kbuild: dwarf: use AS_VERSION instead of test_dwarf5_support.sh · 2e988157
      Masahiro Yamada authored
      The test code in scripts/test_dwarf5_support.sh is somewhat difficult
      to understand, but after all, we want to check binutils >= 2.35.2
      
      From the former discussion, the requirement for generating DWARF v5 from
      C code is as follows:
      
       - gcc + gnu as          -> requires gcc 5.0+ (but 7.0+ for full support)
       - clang + gnu as        -> requires binutils 2.35.2+
       - clang + integrated as -> OK
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Reviewed-by: default avatarNathan Chancellor <nathan@kernel.org>
      Reviewed-by: default avatarNick Desaulniers <ndesaulniers@google.com>
      2e988157
    • Masahiro Yamada's avatar
      kbuild: check the minimum assembler version in Kconfig · ba64beb1
      Masahiro Yamada authored
      Documentation/process/changes.rst defines the minimum assembler version
      (binutils version), but we have never checked it in the build time.
      
      Kbuild never invokes 'as' directly because all assembly files in the
      kernel tree are *.S, hence must be preprocessed. I do not expect
      raw assembly source files (*.s) would be added to the kernel tree.
      
      Therefore, we always use $(CC) as the assembler driver, and commit
      aa824e0c ("kbuild: remove AS variable") removed 'AS'. However,
      we are still interested in the version of the assembler acting behind.
      
      As usual, the --version option prints the version string.
      
        $ as --version | head -n 1
        GNU assembler (GNU Binutils for Ubuntu) 2.35.1
      
      But, we do not have $(AS). So, we can add the -Wa prefix so that
      $(CC) passes --version down to the backing assembler.
      
        $ gcc -Wa,--version | head -n 1
        gcc: fatal error: no input files
        compilation terminated.
      
      OK, we need to input something to satisfy gcc.
      
        $ gcc -Wa,--version -c -x assembler /dev/null -o /dev/null | head -n 1
        GNU assembler (GNU Binutils for Ubuntu) 2.35.1
      
      The combination of Clang and GNU assembler works in the same way:
      
        $ clang -no-integrated-as -Wa,--version -c -x assembler /dev/null -o /dev/null | head -n 1
        GNU assembler (GNU Binutils for Ubuntu) 2.35.1
      
      Clang with the integrated assembler fails like this:
      
        $ clang -integrated-as -Wa,--version -c -x assembler /dev/null -o /dev/null | head -n 1
        clang: error: unsupported argument '--version' to option 'Wa,'
      
      For the last case, checking the error message is fragile. If the
      proposal for -Wa,--version support [1] is accepted, this may not be
      even an error in the future.
      
      One easy way is to check if -integrated-as is present in the passed
      arguments. We did not pass -integrated-as to CLANG_FLAGS before, but
      we can make it explicit.
      
      Nathan pointed out -integrated-as is the default for all of the
      architectures/targets that the kernel cares about, but it goes
      along with "explicit is better than implicit" policy. [2]
      
      With all this in my mind, I implemented scripts/as-version.sh to
      check the assembler version in Kconfig time.
      
        $ scripts/as-version.sh gcc
        GNU 23501
        $ scripts/as-version.sh clang -no-integrated-as
        GNU 23501
        $ scripts/as-version.sh clang -integrated-as
        LLVM 0
      
      [1]: https://github.com/ClangBuiltLinux/linux/issues/1320
      [2]: https://lore.kernel.org/linux-kbuild/20210307044253.v3h47ucq6ng25iay@archlinux-ax161/Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Reviewed-by: default avatarNathan Chancellor <nathan@kernel.org>
      ba64beb1
    • Masahiro Yamada's avatar
      kbuild: collect minimum tool versions into scripts/min-tool-version.sh · e24b3ffc
      Masahiro Yamada authored
      The kernel build uses various tools, many of which are provided by the
      same software suite, for example, LLVM and Binutils.
      
      When you raise the minimum version of Clang/LLVM, you need to update
      clang_min_version in scripts/cc-version.sh and also lld_min_version in
      scripts/ld-version.sh.
      
      Kbuild can handle CC=clang and LD=ld.lld independently, but it does not
      make much sense to maintain their versions separately.
      
      Let's create a central place of minimum tool versions so you do not need
      to touch multiple files. scripts/min-tool-version.sh prints the minimum
      version of the given tool.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      Reviewed-by: default avatarNathan Chancellor <nathan@kernel.org>
      Acked-by: default avatarMiguel Ojeda <ojeda@kernel.org>
      Tested-by: default avatarSedat Dilek <sedat.dilek@gmail.com>
      e24b3ffc
    • Masahiro Yamada's avatar
      kbuild: replace sed with $(subst ) or $(patsubst ) · 6e0839fd
      Masahiro Yamada authored
      For simple text replacement, it is better to use a built-in function
      instead of sed if possible. You can save one process forking.
      
      I do not mean to replace all sed invocations because GNU Make itself
      does not support regular expression (unless you use guile).
      
      I just replaced simple ones.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      6e0839fd
    • Bhaskar Chowdhury's avatar
    • Nathan Chancellor's avatar
      Makefile: Only specify '--prefix=' when building with clang + GNU as · eec08090
      Nathan Chancellor authored
      When building with LLVM_IAS=1, there is no point to specifying
      '--prefix=' because that flag is only used to find GNU cross tools,
      which will not be used indirectly when using the integrated assembler.
      All of the tools are invoked directly from PATH or a full path specified
      via the command line, which does not depend on the value of '--prefix='.
      
      Sharing commands to reproduce issues becomes a little bit easier without
      a '--prefix=' value because that '--prefix=' value is specific to a
      user's machine due to it being an absolute path.
      
      Some further notes from Fangrui Song:
      
        clang can spawn GNU as (if -f?no-integrated-as is specified) and GNU
        objcopy (-f?no-integrated-as and -gsplit-dwarf and -g[123]).
        objcopy is only used for GNU as assembled object files.
        With integrated assembler, the object file streamer creates .o and
        .dwo simultaneously.
        With GNU as, two objcopy commands are needed to extract .debug*.dwo to
        .dwo files && another command to remove .debug*.dwo sections.
      
      A small consequence of this change (to keep things simple) is that
      '--prefix=' will always be specified now, even with a native build, when
      it was not before. This should not be an issue due to the way that the
      Makefile searches for the prefix (based on elfedit's location). This
      ends up improving the experience for host builds because PATH is better
      respected and matches GCC's behavior more closely. See the below thread
      for more details:
      
      https://lore.kernel.org/r/20210205213651.GA16907@Ryzen-5-4500U.localdomain/Signed-off-by: default avatarNathan Chancellor <nathan@kernel.org>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      eec08090
    • Nathan Chancellor's avatar
      Makefile: Remove '--gcc-toolchain' flag · c91d4e47
      Nathan Chancellor authored
      This flag was originally added to allow clang to find the GNU cross
      tools in commit 785f11aa ("kbuild: Add better clang cross build
      support"). This flag was not enough to find the tools at times so
      '--prefix' was added to the list in commit ef8c4ed9 ("kbuild: allow
      to use GCC toolchain not in Clang search path") and improved upon in
      commit ca9b31f6 ("Makefile: Fix GCC_TOOLCHAIN_DIR prefix for Clang
      cross compilation"). Now that '--prefix' specifies a full path and
      prefix, '--gcc-toolchain' serves no purpose because the kernel builds
      with '-nostdinc' and '-nostdlib'.
      
      This has been verified with self compiled LLVM 10.0.1 and LLVM 13.0.0 as
      well as a distribution version of LLVM 11.1.0 without binutils in the
      LLVM toolchain locations.
      
      Link: https://reviews.llvm.org/D97902Signed-off-by: default avatarNathan Chancellor <nathan@kernel.org>
      Reviewed-by: default avatarFangrui Song <maskray@google.com>
      Reviewed-by: default avatarNick Desaulniers <ndesaulniers@google.com>
      Tested-by: default avatarNick Desaulniers <ndesaulniers@google.com>
      Tested-by: default avatarSedat Dilek <sedat.dilek@gmail.com>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      c91d4e47
    • Masahiro Yamada's avatar
      kbuild: move $(strip ) to suffix-search definition · a34e6d1e
      Masahiro Yamada authored
      Move $(strip ...) to the callee from the callers of suffix-search. It
      shortens the code slightly. Adding a space after a comma will not be
      a matter. I also dropped parentheses from single character variables.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      a34e6d1e
    • Masahiro Yamada's avatar
      kbuild: rename multi-used-* to multi-obj-* · a6601e01
      Masahiro Yamada authored
      I think multi-obj-* is clearer, and more consistent with real-obj-*.
      
      Rename as follows:
      
        multi-used-y  ->  multi-obj-y
        multi-used-m  ->  multi-obj-m
        multi-used    ->  multi-obj-ym
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      a6601e01
    • Rasmus Villemoes's avatar
      kbuild: apply fixdep logic to link-vmlinux.sh · 0b956e20
      Rasmus Villemoes authored
      The patch adding CONFIG_VMLINUX_MAP revealed a small defect in the
      build system: link-vmlinux.sh takes decisions based on CONFIG_*
      options, but changing one of those does not always lead to vmlinux
      being linked again.
      
      For most of the CONFIG_* knobs referenced previously, this has
      probably been hidden by those knobs also affecting some object file,
      hence indirectly also vmlinux.
      
      But CONFIG_VMLINUX_MAP is only handled inside link-vmlinux.sh, and
      changing CONFIG_VMLINUX_MAP=n to CONFIG_VMLINUX_MAP=y does not cause
      the build system to re-link (and hence have vmlinux.map
      emitted). Since that map file is mostly a debugging aid, this is
      merely a nuisance which is easily worked around by just deleting
      vmlinux and building again.
      
      But one could imagine other (possibly future) CONFIG options that
      actually do affect the vmlinux binary but which are not captured
      through some object file dependency.
      
      To fix this, make link-vmlinux.sh emit a .vmlinux.d file in the same
      format as the dependency files generated by gcc, and apply the fixdep
      logic to that. I've tested that this correctly works with both in-tree
      and out-of-tree builds.
      Signed-off-by: default avatarRasmus Villemoes <linux@rasmusvillemoes.dk>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      0b956e20
    • Rasmus Villemoes's avatar
      kbuild: add CONFIG_VMLINUX_MAP expert option · 5cc12472
      Rasmus Villemoes authored
      It can be quite useful to have ld emit a link map file, in order to
      debug or verify that special sections end up where they are supposed
      to, and to see what LD_DEAD_CODE_DATA_ELIMINATION manages to get rid
      of.
      
      The only reason I'm not just adding this unconditionally is that the
      .map file can be rather large (several MB), and that's a waste of
      space when one isn't interested in these things. Also make it depend
      on CONFIG_EXPERT.
      Signed-off-by: default avatarRasmus Villemoes <linux@rasmusvillemoes.dk>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      5cc12472
    • Masahiro Yamada's avatar
      kbuild: show warning if 'make headers_check' is used · 609bbb4d
      Masahiro Yamada authored
      Since commit 7ecaf069 ("kbuild: move headers_check rule to
      usr/include/Makefile"), 'make headers_check' is no-op.
      
      This stub target is remaining here in case some scripts still invoke it.
      In order to prompt people to remove stale code, show a noisy warning
      message if used. The stub will be really removed after the Linux 5.15
      release.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      609bbb4d
    • Masahiro Yamada's avatar
      kbuild: include Makefile.compiler only when compiler is needed · 805b2e1d
      Masahiro Yamada authored
      Since commit f2f02ebd ("kbuild: improve cc-option to clean up all
      temporary files"), running 'make kernelversion' in a read-only source
      tree emits a bunch of warnings:
      
        mkdir: cannot create directory '.tmp_12345': Permission denied
      
      No-build targets such as kernelversion, clean, help, etc. do not
      need to evaluate $(call cc-option,) or friends. Skip Makefile.compiler
      so $(call cc-option,) becomes no-op.
      
      This not only fixes the warnings, but also runs non-build targets much
      faster.
      
      Basically, all installation targets should also be non-build targets.
      Unfortunately, vdso_install requires the compiler because it builds
      vdso before installation. This is a problem that must be fixed by a
      separate patch.
      Reported-by: default avatarIsrael Tsadok <itsadok@gmail.com>
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      805b2e1d
    • Masahiro Yamada's avatar
      kbuild: split cc-option and friends to scripts/Makefile.compiler · 57fd251c
      Masahiro Yamada authored
      scripts/Kbuild.include is included everywhere, but macros such as
      cc-option are needed by build targets only.
      
      For example, when 'make clean' traverses the tree, it does not need
      to evaluate $(call cc-option,).
      
      Split cc-option, ld-option, etc. to scripts/Makefile.compiler, which
      is only included from the top Makefile and scripts/Makefile.build.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      57fd251c
  2. 15 Mar, 2021 1 commit
    • Masahiro Yamada's avatar
      kbuild: prefix $(srctree)/ to some included Makefiles · 3204a7fb
      Masahiro Yamada authored
      VPATH is used in Kbuild to make pattern rules search for prerequisites
      in both $(objtree) and $(srctree). Some of *.c, *.S files are not real
      sources, but generated by tools such as flex, bison, perl.
      
      In contrast, I doubt the benefit of --include-dir=$(abs_srctree) because
      it is always clear which Makefiles are real sources, and which are not.
      
      So, my hope is to add $(srctree)/ prefix to all check-in Makefiles,
      then remove --include-dir=$(abs_srctree) flag in the future.
      
      I am touching only some Kbuild core parts for now. Treewide fixes will
      be needed to achieve this goal.
      Signed-off-by: default avatarMasahiro Yamada <masahiroy@kernel.org>
      3204a7fb
  3. 14 Mar, 2021 11 commits