Work space

Let's start out by taking a look at a typically working area. Note that this may not be exactly what see - there are a lot of options that can effect exactly how things are done, but it gives us a pretty good idea of whats going on. What we are looking at here is the tmp directory (as specified by TMPDIR in your local.conf):

~%> find tmp -maxdepth 2 -type d 
tmp
tmp/stamps
tmp/cross
tmp/cross/bin
tmp/cross/libexec
tmp/cross/lib  
tmp/cross/share
tmp/cross/sh4-linux
tmp/cache
tmp/cache/titan
tmp/work
tmp/work/busybox-1.2.1-r13
tmp/work/libice-1_1.0.3-r0 
tmp/work/arpwatch-2.1a15-r2
...
tmp/rootfs
tmp/rootfs/bin
tmp/rootfs/usr
tmp/rootfs/media
tmp/rootfs/dev  
tmp/rootfs/var  
tmp/rootfs/lib  
tmp/rootfs/sbin 
tmp/rootfs/mnt  
tmp/rootfs/boot 
tmp/rootfs/sys  
tmp/rootfs/proc 
tmp/rootfs/etc  
tmp/rootfs/home 
tmp/rootfs/tmp  
tmp/staging     
tmp/staging/man 
tmp/staging/x86_64-linux
tmp/staging/pkgdata
tmp/staging/pkgmaps
tmp/staging/var
tmp/staging/sh4-linux
tmp/staging/local
tmp/staging/etc  
tmp/deploy
tmp/deploy/addons
tmp/deploy/ipk   
tmp/deploy/sources
tmp/deploy/images

The various top level directories under tmp include:

stamps

Nothing of interest to users in here. These time stamps are used by bitbake to keep track of what tasks it has completed and what tasks it still has outstanding. This is how it knows that certain actions have been completed and it doesn't need to do them again.

cross

Contains the cross-compiler toolchain. That is the gcc and binutils that run on the host system but produce output for the target system.

cache

Nothing of interest to users in here. This contains the bitbake parse cache and is used to avoid the need to parse all of the recipes each time bitbake is run. This makes bitbake a lot faster on the 2nd and subsequent runs.

work

The work directory. This is the directory in which all packages are built - this is where the source code is extract, patches applied, software configure, compiled, installed and package. This is where you'll spend most of you time looking when working in OE.

rootfs

The generated root filesystem image for your target device. This is the contents of the root filesystem (NOTE: fakeroot means it doesn't have the correct device special nodes and permissions to use directly).

staging

Contains the staging area, which is used to stored natively compiled tools and and libraries and headers for the target that are required for building other software.

deploy

Contains the final output from OE. This includes the installation packages (typically .ipkg packages) and flash and/or disk images. This is where you go to get the final product.

When people refer to the "tmp directory" this is the directory them are talking about.

To perform a complete rebuild from script you would usually rename or delete tmp and then restart your build. I recommend keeping one old version of tmp around to use for comparison if something goes wrong with your new build. For example:

%> rm -fr tmp.OLD
$> mv tmp tmp.OLD
%> bitbake bootstrap-image

work directory (tmp/work)

The work directory is where all source code is unpacked into, where source is configured, compiled and packaged. In other words this is where all the action happens. Each bitbake recipe will produce a corresponding sub directory in the work directory. The sub directory name will contain the recipe name, version and the release number (as defined by the PR variable within the recipe).

Here's an example of a few of the subdirectories under the work directory:

~%> find tmp/work -maxdepth 1 -type d | head -4
tmp/work
tmp/work/busybox-1.2.1-r13
tmp/work/libice-1_1.0.3-r0
tmp/work/arpwatch-2.1a15-r2

You can see that the first three (of several hundred) recipes here and they are for release 13 of busybox 1.2.1, release 0 or libice 1.1.0.3 and release 2 of arpwatch 2.1a15. It's also possible that you may just have a sub directory for your targets architecture and operating system in which case these directories will be in that additional subdirectory, as shown here:

~%> find tmp/work -maxdepth 2 -type d | head -4
tmp/work
tmp/work/sh4-linux
tmp/work/sh4-linux/busybox-1.2.1-r13
tmp/work/sh4-linux/libice-1_1.0.3-r0
tmp/work/sh4-linux/arpwatch-2.1a15-r2

The sh4-linux directory in the above example is a combination of the target architecture (sh4) and operating system (linux). This subdirectory has been added by the use of one of OpenEmbedded's many features. In this case it's the multimachine feature which is used to allow builds for multiple targets within the one work directory and can be enabled on a per distribution basis. This feature enables the sharing of native and architecture neutral packages and building for multiple targets that support the same architecture but require different linux kernels (for example). We'll assume multimachine isn't being used for the rest of this chapter, just remember to add the extra directory if your distribution is using it.

Using lzo 1.08 as an example we'll examine the contents of the working directory for a typical recipe:

~%> find tmp/work/lzo-1.08-r14 -maxdepth 1
tmp/work/lzo-1.08-r14
tmp/work/lzo-1.08-r14/temp
tmp/work/lzo-1.08-r14/lzo-1.08
tmp/work/lzo-1.08-r14/install
tmp/work/lzo-1.08-r14/image

The directory, tmp/work/lzo-1.08-r14, is know as the "working directory" for the recipe and is specified via the WORKDIR variable in bitbake. You'll sometimes see recipes refer directly to WORKDIR and this is the directory they are referencing. The 1.08 is the version of lzo and r14 is the release number, as defined by the PR variable within the recipe.

Under the working directory (WORKDIR) there are four subdirectories:

temp

The temp directories contains logs and in some cases scripts that actually implement specific tasks (such as a script to configure or compile the source).

You can look at the logs in this directory to get more information into what happened (or didn't happen). This is usually the first thing to look at when things are going wrong and these usually need to be included when reporting bugs.

The scripts can be used to see what a particular task, such as configure or compile, is trying to do.

lzo-1.08

This is the unpacked source code directory, which was created when the lzo source code was extracted in this directory. The name and format of this directory is therefore dependent on the actual source code packaging. Within recipes this directory is referred to as S and is usually expected to be named like this, that is "-". If the source code extracts to somewhere else then that would need to be declared in the recipe by explicitly setting the value of the variable S to the appropriate directory.

image

The image directory (or destination directory) is where the software needs to be installed into in order to be packaged. This directory is referred to as D in recipes. So instead of installing binaries into /usr/bin and libraries into /usr/lib for example you would need to install into ${D}/usr/bin and ${D}/usr/lib instead. When installed on the target the ${D} will be not be included so they'll end up in the correct place. You definitely don't wont files on your host system being replaced by cross-compiled binaries for your target!

install

The install directory is used to split the installed files into separate packages. One subdirectory is created per package to be generated and the files are moved from the image directory (D) over to this directory, and into the appropriate package subdirectory, as each packaging instruction is processed. Typically there will be separate documentation (-doc), debugging (-dbg) and development (-dev) packages automatically created. There are variables such as FILES_ and PACKAGES used in recipes which control the separation of various files into individual packages.

So lets show some examples of the useful information you now have access to.

How about checking out what happened during the configuration of lzo? Well that requires checking the log file for configure that is generated in the temp directory:

~%> less tmp/work/lzo-1.08-r14/temp/log.do_configure.*
...
checking whether ccache sh4-linux-gcc -ml -m4 suffers the -fschedule-insns bug... unknown
checking whether ccache sh4-linux-gcc -ml -m4 suffers the -fstrength-reduce bug... unknown
checking whether ccache sh4-linux-gcc -ml -m4 accepts -fstrict-aliasing... yes
checking the alignment of the assembler... 0
checking whether to build assembler versions... no
configure: creating ./config.status
config.status: creating Makefile
config.status: creating examples/Makefile
config.status: creating include/Makefile
config.status: creating ltest/Makefile
config.status: creating minilzo/Makefile
config.status: creating src/Makefile
config.status: creating tests/Makefile
config.status: creating config.h
config.status: executing depfiles commands

Or perhaps you want to see how the files were distributed into individual packages prior to packaging? The install directory is where the files are split into separate packages and so that shows us which files end up where:

~%> find tmp/work/lzo-1.08-r14/install                
tmp/work/lzo-1.08-r14/install
tmp/work/lzo-1.08-r14/install/lzo-doc
tmp/work/lzo-1.08-r14/install/lzo-dbg
tmp/work/lzo-1.08-r14/install/lzo-dbg/usr
tmp/work/lzo-1.08-r14/install/lzo-dbg/usr/lib
tmp/work/lzo-1.08-r14/install/lzo-dbg/usr/lib/.debug
tmp/work/lzo-1.08-r14/install/lzo-dbg/usr/lib/.debug/liblzo.so.1.0.0
tmp/work/lzo-1.08-r14/install/lzo-dev
tmp/work/lzo-1.08-r14/install/lzo-dev/usr
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include/lzo2a.h
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include/lzo1y.h
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include/lzo1.h
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include/lzo1b.h
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include/lzo1f.h
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include/lzoconf.h
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include/lzo1x.h
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include/lzo16bit.h
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include/lzo1a.h
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include/lzo1z.h
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include/lzoutil.h
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/include/lzo1c.h
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/lib
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/lib/liblzo.a
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/lib/liblzo.so
tmp/work/lzo-1.08-r14/install/lzo-dev/usr/lib/liblzo.la
tmp/work/lzo-1.08-r14/install/lzo.shlibdeps
tmp/work/lzo-1.08-r14/install/lzo-locale
tmp/work/lzo-1.08-r14/install/lzo
tmp/work/lzo-1.08-r14/install/lzo/usr
tmp/work/lzo-1.08-r14/install/lzo/usr/lib
tmp/work/lzo-1.08-r14/install/lzo/usr/lib/liblzo.so.1
tmp/work/lzo-1.08-r14/install/lzo/usr/lib/liblzo.so.1.0.0