From 84a8eafd4cbd103e89194ed1768cc43760ecde3a Mon Sep 17 00:00:00 2001 From: Thomas Petazzoni Date: Fri, 11 Sep 2009 20:40:42 +0200 Subject: [PATCH] documentation: update after major changes * Remove the huge section describing the design of the removed "project" feature * Update all references to the output directory hierarchy to match the new one * Add new sections on how to rebuild single packages and how to create board support * Other updates, clarifications, reorganizations Signed-off-by: Thomas Petazzoni --- docs/buildroot.html | 669 ++++++++++++++++++-------------------------- 1 file changed, 265 insertions(+), 404 deletions(-) diff --git a/docs/buildroot.html b/docs/buildroot.html index 8cd705df44..45876d7b6b 100644 --- a/docs/buildroot.html +++ b/docs/buildroot.html @@ -24,32 +24,30 @@
  • About Buildroot
  • Obtaining Buildroot
  • Using Buildroot
    • -
  • Customizing the target filesystem
    • +
  • Customizing the generated target filesystem
  • Customizing the Busybox configuration
  • Customizing the uClibc configuration
    • +
  • Understanding how to rebuild packages
  • How Buildroot works
    • -
  • Building several projects in the - same buildroot source tree
  • Using the uClibc toolchain outside Buildroot
  • Use an external toolchain
  • Location of downloaded packages
    • -
  • Adapting the rootfs for own binaries
  • Extending Buildroot with more Software
    • +
  • Creating your own board support
  • Resources

    • About Buildroot

    -

    Buildroot is a set of Makefiles and patches that allow to easily - generate both a cross-compilation toolchain and a root filesystem for your - target. The cross-compilation toolchain uses uClibc (http://www.uclibc.org/), a tiny C standard - library.

    +

    Buildroot is a set of Makefiles and patches that allow to + easily generate a cross-compilation toolchain, a root filesystem + and a Linux kernel image for your target. Buildroot can be used + for either one, two or all of these options, independently.

    Buildroot is useful mainly for people working with embedded systems. Embedded systems often use processors that are not the regular x86 @@ -169,10 +167,6 @@

    For each entry of the configuration tool, you can find associated help that describes the purpose of the entry.

    -

    One of the key configuration items is the PROJECT which - determines where some board specific packages are built and where the - results are stored.

    -

    Once everything is configured, the configuration tool has generated a .config file that contains the description of your configuration. It will be used by the Makefiles to do what's needed.

    @@ -184,48 +178,51 @@ $ make -

    This command will download, configure and compile all the selected - tools, and finally generate a target filesystem. The target filesystem will - be named root_fs_ARCH.EXT where ARCH is your - architecture and EXT depends on the type of target filesystem - selected in the Target options section of the configuration - tool. - The file is stored in the "binaries/$(PROJECT)/" directory

    - -

    - Creating your own board support

    - -

    Once a package has been unpacked, it is possible to manually update - configuration files. Buildroot can automatically save the configuration - of buildroot, linux, busybox, uclibc and u-boot in "local/$(PROJECT) by - using the command: -

    - -
    - $ make saveconfig
-
    +

    This command will download, configure and compile all the + selected tools, and finally generate a toolchain, a root + filesystem image and a kernel image (or only one of these + elements, depending on the configuration).

    -

    Once a buildroot configuration has been created by saveconfig, - the default "$(TOPDIR)/.config" file can be overridden by

    +

    Buildroot output is stored in a single directory, + output/. This directory contains several + subdirectories:

    -
    - $ make BOARD=<project>
-
    - -

    Buildroot will then use "local/<project>/<project>.config" - instead of ".config".

    - -

    If you want to modify your board, you can copy the project configuration - file to ".config" by using the command:

    - -
    - $ make BOARD=<project> getconfig
-
    +

    Offline builds

    @@ -259,13 +256,18 @@

    Buildroot optionally honors some environment variables that are passed to make :

      -
    • HOSTCXX
      • -
    • HOSTCC
      • -
    • UCLIBC_CONFIG_FILE=<path/to/.config>
      • -
    • BUSYBOX_CONFIG_FILE=<path/to/.config>
      • -
    • BUILDROOT_COPYTO
      • -
    • BUILDROOT_DL_DIR
      • -
    • BUILDROOT_LOCAL
      • +
    • HOSTCXX, the host C++ compiler to use
      • +
    • HOSTCC, the host C compiler to use
      • +
    • UCLIBC_CONFIG_FILE=<path/to/.config>, path + to the uClibc configuration file to use to compile uClibc if an + internal toolchain is selected
      • +
    • BUSYBOX_CONFIG_FILE=<path/to/.config>, path + to the Busybox configuration file
      • +
    • BUILDROOT_COPYTO, an additional location at which + the binary images of the root filesystem, kernel, etc. built by + Buildroot are copied
      • +
    • BUILDROOT_DL_DIR to override the directory in + which Buildroot stores/retrieves downloaded files

    An example that uses config files located in the toplevel directory and @@ -289,43 +291,14 @@ $ make HOSTCXX=g++-4.3-HEAD HOSTCC=gcc-4.3-HEAD $ export BUILDROOT_COPYTO=/tftpboot - -

    - Using auto-completion

    - -

    If you are lazy enough that you don't want to type the entire make - menuconfig command line, you can enable auto-completion in your shell. - Here is how you can do that using bash:

    -
    -$ complete -W menuconfig make
-
    - -

    Then just enter the beginning of the line, and ask bash to - complete it for you by pressing the TAB key:

    -
    -$ make me<TAB>
-
    - -

    will result in bash to append nuconfig for you!

    - -

    Alternatively, some distributions (of which Debian and Mandriva are but - an example) have more powerful make completion. Depending on you - distribution, you may have to install a package to enable completion. Under - Mandriva, this is bash-completion, while Debian ships it as part of - the bash package.

    - -

    Other shells, such as zsh, also have completion facilities. See - the documentation for your shell.

    -

    Customizing the - target filesystem

    + generated target filesystem

    There are a few ways to customize the resulting target filesystem:

    • Customize the target filesystem directly, and rebuild the image. The - target filesystem is available under project_build_ARCH/root/ - where ARCH is the chosen target architecture. + target filesystem is available under output/target/. You can simply make your changes here, and run make afterwards, which will rebuild the target filesystem image. This method allows to do everything on the target filesystem, but if you decide to completely rebuild your @@ -347,14 +320,31 @@ $ make me<TAB> inside the skeleton because it contains symlinks that would be broken otherwise.
      These customizations are deployed into - project_build_ARCH/root/ just before the actual image + output/target/ just before the actual image is made. So simply rebuilding the image by running make should propagate any new changes to the image.
      • -
    • When configuring the build system, using make - menuconfig or make xconfig, you can specify - the contents of the /etc/hostname and /etc/issue (the welcome - banner) in the PROJECT section
      • +
    • Add support for your own target in Buildroot so that you + have your own target skeleton, see this + section for details
      • + +
    • In Buildroot configuration, you can specify the path to a + post-build script that gets called after Buildroot built + all the selected software, but before the the rootfs + packages are assembled. The destination root filesystem folder + is given as first argument to this script, and this script can + then be used to copy programs, static data or any other needed + file to your target filesystem.
      You should, however, use + that feature with care. Whenever you find that a certain package + generates wrong or unneeded files, you should rather fix than + package than working around it with a cleanup script.
      • + +
    • A special package, customize, stored in + package/customize can be used. You can put all the + files that you want to see in the final target root filesystem + in package/customize/source, and then enable this + special package from the configuration system.
      • +

    Customizing the @@ -423,6 +413,73 @@ $ make me<TAB>

    If you want to use an existing config file for uclibc, then see section environment variables.

    +

    Understanding how to rebuild + packages

    + +

    One of the most common question and issue about Buildroot + encountered by users is how to rebuild a given package or how to + remove a package without rebuilding everything from scratch.

    + +

    Removing a package is currently unsupported by Buildroot + without rebuilding from scratch. This is because Buildroot doesn't + keep track of which package installs what files in the + output/staging and output/target + directories. However, implement clean package removal is on the + TODO-list of Buildroot developers.

    + +

    To rebuild a single package from scratch, the easiest way is to + remove its build directory in output/build. Buildroot + will then re-extract, re-configure, re-compile and re-install this + package from scratch.

    + +

    However, if you don't want to rebuild the package completely + from scratch, a better understanding of the Buildroot internals is + needed. Internally, to keep track of which steps have been done + and which steps remains to be done, Buildroot maintains stamps + files (i.e, empty files that just tell whether this or this action + has been done). The problem is that these stamps files are not + uniformely named and handled by the different packages, so some + understanding of the particular package is needed.

    + +

    For packages relying on the autotools Buildroot + infrastructure (see this section for + details), the following stamps files are interesting:

    + +
      + +
    • output/build/packagename-version/.stamp_configured. If + removed, Buildroot will trigger the recompilation of the package + from the configuration step (execution of + ./configure)
      • + +
    • output/build/packagename-version/.stamp_built. If + removed, Buildroot will trigger the recompilation of the package + from the compilation step (execution of make)
      • + +
    + +

    For other packages, an analysis of the specific + package.mk file is needed. For example, the zlib Makefile + looks like:

    + + 
    +$(ZLIB_DIR)/.configured: $(ZLIB_DIR)/.patched
+        (cd $(ZLIB_DIR); rm -rf config.cache; \
+                        [...]
+        )
+        touch $@
+
+$(ZLIB_DIR)/libz.a: $(ZLIB_DIR)/.configured
+        $(MAKE) -C $(ZLIB_DIR) all libz.a
+        touch -c $@
+
    + +

    So, if you want to trigger the reconfiguration, you need to + remove output/build/zlib-version/.configured and if + you want to trigger only the recompilation, you need to remove + output/build/zlib-version/libz.a.

    +

    How Buildroot works

    @@ -477,334 +534,154 @@ $ make me<TAB> configuration is done) :

      -
    1. Create the download directory (dl/ by default). This is - where the tarballs will be downloaded. It is interesting to know that the - tarballs are in this directory because it may be useful to save them - somewhere to avoid further downloads.
      2. - -
    2. Create the shared build directory (build_ARCH/ by - default, where ARCH is your architecture). This is where all - non configurable user-space tools will be compiled.When building two or - more targets using the same architecture, the first build will go through - the full download, configure, make process, but the second and later - builds will only copy the result from the first build to its project - specific target directory significantly speeding up the build process
      3. - -
    3. Create the project specific build directory - (project_build_ARCH/$(PROJECT) by default, where - ARCH is your architecture). This is where all configurable - user-space tools will be compiled. The project specific build directory - is neccessary, if two different targets needs to use a specific package, - but the packages have different configuration for both targets. Some - examples of packages built in this directory are busybox and linux. -
      4. -
    4. Create the project specific result directory - (binaries/$(PROJECT) by default, where ARCH - is your architecture). This is where the root filesystem images are - stored, It is also used to store the linux kernel image and any - utilities, boot-loaders etc. needed for a target. -
      5. +
    5. Create all the output directories: staging, + target, build, stamps, + etc. in the output directory (output/ by default, + another value can be specified using O=)
      6. + +
    6. Generate all the targets listed in the + BASE_TARGETS variable. When an internal toolchain + is used, it means generating the cross-compilation + toolchain. When an external toolchain is used, it means checking + the features of the external toolchain and importing it into the + Buildroot environment.
      7. + +
    7. Generate all the targets listed in the TARGETS + variable. This variable is filled by all the individual + components Makefiles. So, generating all these targets will + trigger the compilation of the userspace packages (libraries, + programs), the kernel, the bootloader and the generation of the + root filesystem images, depending on the configuration.
      8. -
    8. Create the toolchain build directory - (toolchain_build_ARCH/ by default, where ARCH - is your architecture). This is where the cross compilation toolchain will - be compiled.
      9. - -
    9. Setup the staging directory (build_ARCH/staging_dir/ by - default). This is where the cross-compilation toolchain will be - installed. If you want to use the same cross-compilation toolchain for - other purposes, such as compiling third-party applications, you can add - build_ARCH/staging_dir/usr/bin to your PATH, and then use - arch-linux-gcc to compile your application. In order to - setup this staging directory, it first removes it, and then it creates - various subdirectories and symlinks inside it.
      10. - -
    10. Create the target directory (project_build_ARCH/root/ by - default) and the target filesystem skeleton. This directory will contain - the final root filesystem. To setup it up, it first deletes it, then it - uncompress the target/generic/skel.tar.gz file to create the - main subdirectories and symlinks, copies the skeleton available in - target/generic/target_skeleton and then removes useless - .svn/CVS directories.
      11. - -
    11. Add the TARGETS dependency. This should generally check - if the configuration option for this package is enabled, and if so then - "subscribe" this package to be compiled by adding it to the - TARGETS global variable.
    -

    Building several - projects in the same buildroot source tree

    - -

    Note: the contents of this section are obsolete since this - feature has been implemented.

    - -

    Background

    - -

    Buildroot has always supported building several projects in the same - tree if each project was for a different architecture.

    +

    + Creating your own board support

    -

    The root file system has been created in the - "build_<ARCH>/root" - directory which is unique for each architecture. - Toolchains have been built in - "toolchain_build_<ARCH>".

    +

    Creating your own board support in Buildroot allows you to have + a convenient place to store the Busybox, uClibc, kernel + configurations, your target filesystem skeleton, and a Buildroot + configuration that match your project.

    -

    It the user wanted to build several root file systems for the same - architecture, a prefix or suffix could be added in the configuration file - so the root file system would be built in - "<PREFIX>_build_<ARCH>_<SUFFIX>/root" - By supplying unique combinations of - "<PREFIX>" and - "<SUFFIX>" - each project would get a unique root file system tree.

    +

    Follow these steps to integrate your board in Buildroot:

    -

    The disadvantage of this approach is that a new toolchain was - built for each project, adding considerable time to the build - process, even if it was two projects for the same chip.

    - -

    This drawback has been somewhat lessened with - gcc-4.x.y which allows buildroot to use an external - toolchain. Certain packages requires special - features in the toolchain, and if an external toolchain is selected, - this may lack the neccessary features to complete the build of the root - file system.

    - -

    A bigger problem was that the - "build_<ARCH>" tree - was also duplicated, so each package would also - be rebuilt once per project, resulting in even longer build times.

    - -

    Project to share toolchain and package builds

    - -

    Work has started on a project which will allow the user to build - multiple root file systems for the same architecture in the same tree. - The toolchain and the package build directory will be shared, but each - project will have a dedicated directory tree for project specific - builds.

    - -

    With this approach, most, if not all packages will be compiled - when the first project is built. - The process is almost identical to the original process. - Packages are downloaded and extracted to the shared - "build_<ARCH>/<package>" - directory. They are configured and compiled.

    - -

    Package libraries and headers are installed in the shared $(STAGING_DIR), - and then the project specific root file system "$(TARGET_DIR)" - is populated.

    - -

    At the end of the build, the root file system will be used - to generate the resulting root file system binaries.

    - -

    Once the first project has been built, building other projects will - typically involve populating the new project's root file system directory - from the existing binaries generated in the shared - "build_<ARCH>/<>" directory.

    - -

    Only packages, not used by the first project, will have to go - through the normal extract-configure-compile flow.

    - -

    Implementation

    - -

    The core of the solution is the introduction - of two new directories:

    - - +
  • Create a new directory in target/device/, named + after your company or organization
    • -

    Each of the directories contain one subdirectory per project. - The name of the subdirectory is configured by the user in the - normal buildroot configuration, using the value of:

    +
  • Add a line source + "target/device/yourcompany/Config.in" in + target/device/Config.in so that your board appears + in the configuration system
    • -

    Project Options ---> Project name

    +
  • In target/device/yourcompany/, create a + directory for your project. This way, you'll be able to store + several projects of your company/organization inside + Buildroot.
    • -

    The configuration defines the $(PROJECT) variable.

    +
  • Create a target/device/yourcompany/Config.in + file that looks like the following: -

    The default project name is "uclibc".

    + 
    +menuconfig BR2_TARGET_COMPANY
+       bool "Company projects"
 
-    
    "package/Makefile.in" defines:
-    
    -    PROJECT_BUILD_DIR:=project_build_$(ARCH)/$(PROJECT)
-    BINARIES_DIR:=binaries/$(PROJECT)
-    
    
-    
    
-
-    
    It also defines the location for the target root file system:
-    
    -    TARGET_DIR:=$(PROJECT_BUILD_DIR)/$(PROJECT)/root
-    
    
-    
    
+if BR2_TARGET_COMPANY
 
-    
    I.E: If the user has choosen
-    "myproject"
-    as the $(PROJECT) name:
+config BR2_TARGET_COMPANY_PROJECT_FOOBAR
+       bool "Support for Company project Foobar"
+       help
+         This option enables support for Company project Foobar
 
-    
      
-    
    • "project_build_<ARCH>/myproject"
      • 
-    
    • "binaries/myproject"
      • 
-    
    
-
-    
    will be created. 
    
-
-    
    Currently, the root file system, busybox and an Atmel
-    customized version of
-    U-Boot, as well as some Atmel specific
-    bootloaders like at91-bootstrap and dataflashboot.bin
-    are built in
-    "$(PROJECT_BUILD_DIR)"
-
-    
    The resulting binaries for all architectures are stored in the
-    "$(BINARIES_DIR)" directory. 
    
-
-    
    Summary
    
+endif
+
    -

    The project will share directories which can be share without - conflicts, but will use unique build directories, where the user - can configure the build.

    + Of course, customize the different values to match your + company/organization and your project. This file will create a + menu entry that contains the different projects of your + company/organization.
    • -

    Linux

    +
  • Create a target/device/yourcompany/Makefile.in + file that looks like the following: -

    The user can select from three different Linux strategies: + 

    +ifeq ($(BR2_TARGET_COMPANY_PROJECT_FOOBAR),y)
+include target/device/yourcompany/project-foobar/Makefile.in
+endif
+
    +
    • - +
  • Now, create the + target/device/yourcompany/project-foobar/Makefile.in + file. It is first recommended to define a + BOARD_PATH variable set to + target/device/yourcompany/project-foobar, as it + will simplify further definitions. Then, the file might define + one or several of the following variables: + +
      + +
    • TARGET_SKELETON to a directory that contains + the target skeleton for your project. If this variable is + defined, this target skeleton will be used instead of the + default one. If defined, the convention is to define it to + $(BOARD_PATH)/target_skeleton, so that the target + skeletonn is stored in the board specific directory.
      • + +
    • TARGET_DEVICE_TABLE to a file that contains + the target device table, i.e the list of device files (in + /dev/) created by the root filesystem building + procedure. If this variable is defined, the given device table + will be used instead of the default one. If defined, the + convention is to define it to + $(BOARD_PATH)/target_device_table.txt. See + target/generic/device_table.txt for an example + file.
      • + +
    -

    The current kernel patches can be applied to the - linux source tree even if the version differs from the - kernel header version.

    +
    • -

    Since the user can select any kernel-patch - he/she will be able to select a non-working combination. - If the patch fails, the user will have to generate a new - proprietary kernel-patch or decide to not apply the kernel - patches

    +
  • Then, in the + target/device/yourcompany/project-foobar/ + directory, you can store different files: -

    There is also support for board specific and - architecture specific patches.

    +
      -

      There will also be a way for the user to supply absolute - or relative paths to patches, possibly outside the main tree. - This can be used to apply custom kernel-header-patches, if - the versions available in buildroot cannot be applied to the - specific linux version used

      +
    • One or more Buildroot configurations, under file named + something_defconfig. Your users will then be able + to run make something_defconfig and get the right + configuration for your project
      • -

      Maybe, there will also be a possibility to supply an - "URL" to a patch available on Internet.

      +
    • Configuration files for the kernel, for Busybox or + uClibc. These files can then be referenced by the Buildroot + configuration described above
      • -

      - If there is no linux config file available, - buildroot starts the linux configuration system, which - defaults to "make menuconfig". -

      +
    -

    Todo

    -
      + -
    1. Configurable packages
      2. -

      Many packages can, on top of the simple - "enable/disable build", - be further configured using Kconfig. - Currently these packages will be compiled using the - configuration specified in the - ".config" file of the first - project demanding the build of the package.

      - -

      If another project uses the same packages, but with - a different configuration,these packages will not be rebuilt, - and the root file system for the new project will be populated - with files from the build of the first project

      - -

      If multiple project are built, and a specific package - needs two different configuration, then the user must - delete the package from the - "build_<ARCH>" directory - before rebuilding the new project.

      - -

      A long term solution is to edit the package makefile and move - the build of the configurable packages from - "build_<ARCH>" to - "project_build_<ARCH>/<project name>" - and send a patch to the buildroot mailing list. - -

    2. Naming conventions
      3. - -

      Names of resulting binaries should reflect the - "project name" - -

    3. Generating File System binaries
      4. -

      - Packages which needs to be installed with the "root" - as owner, will generate a - ".fakeroot.<package>" file - which will be used for the final build of the root file system binary.

      - -

      This was previously located in the - "$(STAGING_DIR)" directory, but was - recently moved to the - "$(PROJECT_BUILD_DIR)" directory.

      - -

      Currently only three packages: - "at", - "ltp-testsuite" and - "nfs-utils" - requests fakeroot.

      - -

      The makefile fragments for each file system type like - "ext2", - "jffs2" or - "squashfs" - will, when the file system binary is generated, - collect all present - ".fakeroot.<package>" files - to a single "_fakeroot.<file system>" - file and call fakeroot.

      - ".fakeroot.<package>" - files are deleted as the last action of the Buildroot Makefile.

      - -

      It needs to be evaluated if any further action for the - file system binary build is needed.

      - -
    +

    Using the - uClibc toolchain outside Buildroot

    + generated toolchain outside Buildroot

    You may want to compile your own programs or other software that are not packaged in Buildroot. In order to do this, you can use the toolchain that was generated by Buildroot.

    The toolchain generated by Buildroot by default is located in - build_ARCH/staging_dir/. The simplest way to use it - is to add build_ARCH/staging_dir/usr/bin/ to your PATH + output/staging/. The simplest way to use it + is to add output/staging/usr/bin/ to your PATH environnement variable, and then to use - arch-linux-gcc, arch-linux-objdump, - arch-linux-ld, etc.

    - -

    For example, you may add the following to your - .bashrc (considering you're building for the MIPS - architecture and that Buildroot is located in - ~/buildroot/) :

    - -
    -export PATH="$PATH:~/buildroot/build_mips/staging_dir/usr/bin/"
-
    + ARCH-linux-gcc, ARCH-linux-objdump, + ARCH-linux-ld, etc.

    -

    Then you can simply do :

    - -
    -mips-linux-gcc -o foo foo.c
-
    +

    The easiest way is of course to add the + output/staging/usr/bin/ directory to your PATH + environment variable.

    Important : do not try to move a gcc-3.x toolchain to an other directory, it won't work. There are some hardcoded paths in the @@ -815,8 +692,8 @@ mips-linux-gcc -o foo foo.c might be cumbersome.

    It is also possible to generate the Buildroot toolchain in - another directory than build_ARCH/staging_dir using - the Build options -> Toolchain and header file + another directory than build/staging using the + Build options -> Toolchain and header file location option. This could be useful if the toolchain must be shared with other users.

    @@ -890,22 +767,6 @@ It allows to generate toolchains based on uClibc, glibc and eglibc for a wide range of architectures, and has good community support.

    -

    Adapting the rootfs - for own binaries

    - -

    When using BR2 in an environment where own software binaries or - static data should be part of the generated rootfs package, the - BR2_ROOTFS_POST_BUILD_SCRIPT feature might interest - you. You can specify a command here which is called after BR2 - built all the selected software, but before the the rootfs - packages are assembled. The destination rootfs folder is given as - first argument. You can add own components here, change - default configurations and remove unwanted files.

    -

    You should, however, use that feature with care. Whenever you - find that a certain package generates wrong or unneeded files, you - should rather fix than package than working around it with a - cleanup script.

    -

    Extending Buildroot with more software

    @@ -992,7 +853,7 @@ endif

    On line 9, we tell Buildroot to install the application to the staging directory. The staging directory, - located in build_ARCH/staging_dir/ is the directory + located in output/staging/ is the directory where all the packages are installed, including their documentation, etc. By default, packages are installed in this location using the make install command.

    -- 2.30.2