Directories: What goes where

A large part of the work or a recipe is involved with specifying where files and found and where they have to go. It's important for example that programs do not try and use files from /usr/include or /usr/lib since they are for the host system, not the target. Similarly you don't want programs installed into /usr/bin since that may overwrite your host system programs with versions that don't work on the host!

The following are some of the directories commonly referred to in recipes and will be described in more detail in the rest of this section:

Working directory: WORKDIR

This working directory for a recipe is where archive files will be extracted, plain files will be placed, subdirectories for logs, installed files etc will be created.

Unpacked source code directory: S

This is where patches are applied and where the program is expected to be compiled in.

Destination directory: D

The destination directory. This is where your package should be installed into. The packaging system will then take the files from directories under here and package them up for installation on the target.

Installation directories: bindir, docdir, ...

There are a set of variables available to describe all of the paths on the target that you may want to use. Recipes should use these variables rather than hard coding any specific paths.

Staging directories: STAGING_LIBDIR, STAGING_INCDIR, ...

Staging directories are a special area for headers, libraries and other files that are generated by one recipe that may be needed by another recipe. A library package for example needs to make the library and headers available to other recipes so that they can link against them.

File path directories: FILE, FILE_DIRNAME, FILESDIR, FILESPATH

These directories are used to control where files are found. Understanding these can help you separate patches for different versions or releases of your recipes and/or use the same patch over multiple versions etc.

WORKDIR: The working directory

The working directory is where the source code is extracted, to which plain files (not patches) are copied and where the logs and installation files are created. A typical reason for needing to reference the work directory is for the handling of non patch files.

If we take a look at the recipe for quagga we can see an example non patch files for configuration and init scripts:

SRC_URI = "http://www.quagga.net/download/quagga-${PV}.tar.gz \
           file://fix-for-lib-inpath.patch;patch=1 \
           file://quagga.init \
           file://quagga.default \
           file://watchquagga.init \
           file://watchquagga.default"

The recipe has two init files and two configuration files, which are not patches, but are actually files that it wants to include in the generated packages. Bitbake will copy these files into the work directory. So to access them during the install task we refer to them via the WORKDIR variable:

do_install () {
    # Install init script and default settings
    install -m 0755 -d ${D}${sysconfdir}/default ${D}${sysconfdir}/init.d ${D}${sysconfdir}/quagga
    install -m 0644 ${WORKDIR}/quagga.default ${D}${sysconfdir}/default/quagga
    install -m 0644 ${WORKDIR}/watchquagga.default ${D}${sysconfdir}/default/watchquagga
    install -m 0755 ${WORKDIR}/quagga.init ${D}${sysconfdir}/init.d/quagga
    install -m 0755 ${WORKDIR}/watchquagga.init ${D}${sysconfdir}/init.d/watchquagga
    ...

S: The unpacked source code directory

Bitbake expects to find the extracted source for a package in a directory called - in the WORKDIR directory. This is the directory in which it will change into before patching, compiling and installating the package.

For example, we have a package called widgets_1.2.bb which we are extracting from the widgets-1.2.tar.gz file. Bitbake expects the source to end up in a directory called widgets-1.2 within the work directory. If the source does not end up in this directory then bitbake needs to be told this by explicitly setting S.

If widgets-1.2.tar.gz actually extracts into a directory called widgets, without the version number, instead of widgets-1.2 then the S variable will be wrong and patching and/or compiling will fail. Therefore we need to override the default value of S to specify the directory the source was actually extracted into:

SRC_URI = "http://www.example.com/software/widgets-${PN}.tar.gz"
S = "${WORKDIR}/widgets"

D: The destination directory

The destination directory is where the completed application and all of it's files are installed into in preparation for packaging. Typically an installation would places files in directories such as /etc and /usr/bin by default. Since those directories are used by the host system we do not want the packages to install into those locations. Instead they need to install into the directories below the destination directory.

So instead of installing into /usr/bin the package needs to install into ${D}/usr/bin.

The following example from arpwatch shows the make install command being passed a ${D} as the DESTDIR variable to control where the makefile installs everything:

do_install() {
        ...
        oe_runmake install DESTDIR=${D}

The following example from quagga shows the use of the destination directory to install the configuration files and init scripts for the package:

do_install () {
        # Install init script and default settings
        install -m 0755 -d ${D}${sysconfdir}/default ${D}${sysconfdir}/init.d ${D}${sysconfdir}/quagga
        install -m 0644 ${WORKDIR}/quagga.default ${D}${sysconfdir}/default/quagga
        install -m 0755 ${WORKDIR}/quagga.init ${D}${sysconfdir}/init.d/quagga

Note

You should not use directories such as /etc and /usr/bin directly in your recipes. You should use the variables that define these locations. The full list of these variables can be found in the Installation directories section of the reference chapter.

Staging directories

Staging is used to make libraries, headers and binaries available for the build of one recipe for use by another recipe. Building a library for example requires that packages be created containing the libraries and headers for development on the target as well as making them available on the host for building other packages that need the libraries and headers.

Making the libraries, headers and binaries available for use by other recipes on the host is called staging and is performed by the stage task in the recipe. Any recipes that contain items that are required to build other packages should have a stage task to make sure the items are all correctly placed into the staging area. The following example from clamav show the clamav library and header being placed into the staging area:

do_stage () {
        oe_libinstall -a -so libclamav ${STAGING_LIBDIR}
        install -m 0644 libclamav/clamav.h ${STAGING_INCDIR}
}

The following from the p3scan recipe show the path to the clamav library and header being passed to the configure script. Without this the configure script would either fail to find the library, or worse still search the host systems directories for the library. Passing in the location results in it searching the correct location and finding the clamav library and headers:

EXTRA_OECONF = "--with-clamav=${STAGING_LIBDIR}/.. \
                --with-openssl=${STAGING_LIBDIR}/.. \
                --disable-ripmime"

While the staging directories are automatically added by OpenEmbedded to the compiler and linking commands it is sometimes necessary, as in the p3scan example above, to explicitly specify the location of the staging directories. Typically this is needed for autoconf scripts that search in multiple places for the libraries and headers.

Note

Many of the helper classes, such as pkgconfig and autotools add appropriate commands to the stage task for you. Check with the individual class descriptions in the reference section to determine what each class is staging automatically for you.

A full list of staging directories can be found in the Staging directories section in the reference chapter.

FILESPATH/FILESDIR: Finding local files

The file related variables are used by bitbake to determine where to look for patches and local files.

Typically you will not need to modify these, but it is useful to be aware of the default values. In particular when searching for patches and/or files (file:// URI's), the default search path is:

${FILE_DIRNAME}/${PF}

This is the package name, version and release, such as "strace-4.5.14-r1". This is very rarely used since the patches would only be found for the one exact release of the recipe.

${FILE_DIRNAME}/${P}

This is the package name and version, such as "strace-4.5.14". This is by far the most common place to place version specified patches.

${FILE_DIRNAME}/${PN}

This is the package name only, such as "strace". This is not commonly used.

${FILE_DIRNAME}/files

This is just the directory called "files". This is commonly used for patches and files that apply to all version of the package.

${FILE_DIRNAME}/

This is just the base directory of the recipe. This is very rarely used since it would just clutter the main directory.

Each of the paths is relative to ${FILE_DIRNAME} which is the directory in which the recipe that is being processed is located.

The full set of variables that control the file locations and patch are:

FILE

The path to the .bb file which is currently being processed.

FILE_DIRNAME

The path to the directory which contains the FILE which is currently being processed.

FILE_DIRNAME = "${@os.path.dirname(bb.data.getVar('FILE', d))}"
FILESPATH

The default set of directories which are available to use for the file:// URI's. Each directory is searched, in the specified order, in an attempt to find the file specified by each file:// URI:

FILESPATH = "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:\
${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}"
FILESDIR

The default directory to search for file:// URI's. Only used if the file is not found in FILESPATH. This can be used to easily add one additional directory to the search path without having to modify the default FILESPATH setting. By default this is just the first directory from FILESPATH.

FILESDIR = "${@bb.which(bb.data.getVar('FILESPATH', d, 1), '.')}" 

Sometimes recipes will modify the FILESPATH or FILESDIR variables to change the default search path for patches and files. The most common situation in which this is done is when one recipe includes another one in which the default values will be based on the name of the package doing the including, not the included package. Typically the included package will expect the files to be located in a directories based on it's own name.

As an example the m4-native recipe includes the m4 recipe. This is fine, except that the m4 recipes expects its files and patches to be located in a directory called m4 directory while the native file name results in them being searched for in m4-native. So the m4-native recipe sets the FILESDIR variable to the value that of m4 to add the actual m4 directory (where m4 itself has its files stored) to the list of directories search for:

        include m4_${PV}.bb
        inherit native
        FILESDIR = "${@os.path.dirname(bb.data.getVar('FILE',d,1))}/m4"