When you go about building and installing a software package there are a number of tasks that you generally follow with most software packages. You probably need to start out by downloading the source code, then unpacking the source code. Maye you need to apply some patches for some reason. Then you might run the configure script of the package, perhaps passing it some options to configure it to your liking. The you might run "make install" to install the software. If your actually going to make some packages, such as .deb or .rpm, then you'd have additional tasks you'd perform to make them.
You find that building things in OpenEmbedded works in a similar way - there are a number of tasks that are executed in a predefined order for each recipe. Any many of the tasks correspond to those listed above like "download the source". In fact you've probably already seen some of the names of these tasks - bitbake displays them as they are processed:
~%> bitbake lzo NOTE: Psyco JIT Compiler (http://psyco.sf.net) not available. Install it to increase performance. NOTE: Handling BitBake files: \ (4541/4541) [100 %] NOTE: Parsing finished. 4325 cached, 0 parsed, 216 skipped, 0 masked. NOTE: build 200705041709: started OE Build Configuration: BB_VERSION = "1.8.2" OE_REVISION = "" TARGET_ARCH = "sh4" TARGET_OS = "linux" MACHINE = "titan" DISTRO = "erouter" DISTRO_VERSION = "0.1-20070504" TARGET_FPU = "" NOTE: Resolving missing task queue dependencies NOTE: preferred version 2.5 of glibc not available (for item virtual/sh4-linux-libc-for-gcc) NOTE: Preparing Runqueue NOTE: Executing runqueue NOTE: Running task 208 of 226 (ID: 11, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_fetch) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_fetch: started NOTE: package lzo-1.08-r14: task do_fetch: completed NOTE: package lzo-1.08: completed NOTE: Running task 209 of 226 (ID: 2, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_unpack) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_unpack: started NOTE: Unpacking /home/lenehan/devel/oe/sources/lzo-1.08.tar.gz to /home/lenehan/devel/oe/build/titan-glibc-25/tmp/work/lzo-1.08-r14/ NOTE: package lzo-1.08-r14: task do_unpack: completed NOTE: package lzo-1.08: completed NOTE: Running task 216 of 226 (ID: 3, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_patch) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_patch: started NOTE: package lzo-1.08-r14: task do_patch: completed NOTE: package lzo-1.08: completed NOTE: Running task 217 of 226 (ID: 4, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_configure) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_configure: started NOTE: package lzo-1.08-r14: task do_configure: completed NOTE: package lzo-1.08: completed NOTE: Running task 218 of 226 (ID: 12, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_qa_configure) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_qa_configure: started NOTE: Checking sanity of the config.log file NOTE: package lzo-1.08-r14: task do_qa_configure: completed NOTE: package lzo-1.08: completed NOTE: Running task 219 of 226 (ID: 0, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_compile) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_compile: started NOTE: package lzo-1.08-r14: task do_compile: completed NOTE: package lzo-1.08: completed NOTE: Running task 220 of 226 (ID: 1, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_install) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_install: started NOTE: package lzo-1.08-r14: task do_install: completed NOTE: package lzo-1.08: completed NOTE: Running task 221 of 226 (ID: 5, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_package) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_package: started NOTE: DO PACKAGE QA NOTE: Checking Package: lzo-dbg NOTE: Checking Package: lzo NOTE: Checking Package: lzo-doc NOTE: Checking Package: lzo-dev NOTE: Checking Package: lzo-locale NOTE: DONE with PACKAGE QA NOTE: package lzo-1.08-r14: task do_package: completed NOTE: package lzo-1.08: completed NOTE: Running task 222 of 226 (ID: 8, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_package_write) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_package_write: started Packaged contents of lzo-dbg into /home/lenehan/devel/oe/build/titan-glibc-25/tmp/deploy/ipk/sh4/liblzo-dbg_1.08-r14_sh4.ipk Packaged contents of lzo into /home/lenehan/devel/oe/build/titan-glibc-25/tmp/deploy/ipk/sh4/liblzo1_1.08-r14_sh4.ipk NOTE: Not creating empty archive for lzo-doc-1.08-r14 Packaged contents of lzo-dev into /home/lenehan/devel/oe/build/titan-glibc-25/tmp/deploy/ipk/sh4/liblzo-dev_1.08-r14_sh4.ipk NOTE: Not creating empty archive for lzo-locale-1.08-r14 NOTE: package lzo-1.08-r14: task do_package_write: completed NOTE: package lzo-1.08: completed NOTE: Running task 223 of 226 (ID: 6, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_populate_staging) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_populate_staging: started NOTE: package lzo-1.08-r14: task do_populate_staging: completed NOTE: package lzo-1.08: completed NOTE: Running task 224 of 226 (ID: 9, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_qa_staging) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_qa_staging: started NOTE: QA checking staging NOTE: package lzo-1.08-r14: task do_qa_staging: completed NOTE: package lzo-1.08: completed NOTE: Running task 225 of 226 (ID: 7, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_distribute_sources) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_distribute_sources: started NOTE: package lzo-1.08-r14: task do_distribute_sources: completed NOTE: package lzo-1.08: completed NOTE: Running task 226 of 226 (ID: 10, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_build) NOTE: package lzo-1.08: started NOTE: package lzo-1.08-r14: task do_build: started NOTE: package lzo-1.08-r14: task do_build: completed NOTE: package lzo-1.08: completed NOTE: Tasks Summary: Attempted 226 tasks of which 213 didn't need to be rerun and 0 failed. NOTE: build 200705041709: completed
The output may look different depending on the version of bitbake being used, and some tasks are only run when specific options are enabled in your distribution. The important point to note is that the various tasks are being run and bitbake shows you each time it starts and completes a task.
So there's a set of tasks here which are being run to generate the final packages. And if you'll notice that every recipe runs through the same set of tasks (ok I'll admit that it is possible that some additional tasks could be run for some recipes, but we'll talk about that later). The tasks that you'll need to be most familiar with are:
The fetch task is responsible for fetching any source code that is required. This means things such as downloading files and checking out from source control repositories such as git or svn.
The unpack task is responsible for extracting files from archives, such as .tar.gz, into the working area and copying any additional files, such as init scripts, into the working area.
The patch task is responsible for applying any patches to the unpacked source code
The configure task takes care of the
configuration of the package. Running a configure script
("./configure
The compile task actually compiles the software. This could be as simple as running make.
The populate_staging task (stage is an alternate, easier to type name, that can be used to refer to this task) is responsible for making available libraries and headers (if any) that may be required by other packages to build. For example if you compile zlib then it's headers and the library need to be made available for other applications to include and link against.
This is different to the install task in that this is responsible for making available libraries and headers for use during build on the development host. Therefore it's libraries which normal have to stage things while applications normally don't need to. The install task on the other hand is making files available for packaging and ultimately installation on the target.
The install task is responsible for actually installing everything. Now this needs to install the software into the destination directory, D. This directory won't actually be a part of the final package though. In other words if you install something into ${D}/bin then it will end up in the /bin directory in the package and therefore on the target.
The package task takes the installed files and splits them into separate directories under the ${WORKDIR}/install directory, one per package. It moves the files for the destination directory, ${D}, that they were installed in into the appropriate packages subdirectory. Usually there will be a main package a separate documentation (-doc), development (-dev) and debugging packages (-dbg) for example.
The package_write task is responsible for taking each packages subdirectory and creating any actual installation package, such as .ipk, .deb or .rpm. Currently .ipk is the only fully supported packing format although .deb packages are being actively worked on. It should be reasonably easy for an experienced OpenEmbedded developer to add support for any other packaging formats they might required.
You'll notice that the bitbake output had tasks prefixed with do_, as in do_install vs install. This is slightly confusing but any task x is implemented via a function called do_x in the class or recipe where it is defined. See places refer to the tasks via their name only and some with the do prefix.
You will almost certainly notice tasks beyond these ones - there are various methods available to insert additional tasks into the tasks sequence. As an example the insane.bbclass, which performs various QA checks, does these checks by inserting a new task called qa_configure between the configure and compile tasks and another new task called qa_staging between populate_staging and build tasks. The former validates the result of the configure task and the late the results of the populate_staging task.
To determine the full list of tasks available for a specific recipe you can run bitbake on the recipe and asking it for the full list of available tasks:
~%> bitbake -b packages/perl/perl_5.8.8.bb -c listtasks NOTE: package perl-5.8.8: started NOTE: package perl-5.8.8-r11: task do_listtasks: started do_fetchall do_listtasks do_rebuild do_compile do_build do_populate_staging do_mrproper do_fetch do_configure do_clean do_package do_unpack do_install do_package_write do_distribute_sources do_showdata do_qa_configure do_qa_staging do_patch NOTE: package perl-5.8.8-r11: task do_listtasks: completed NOTE: package perl-5.8.8: completed ~%>
If your being observant you'll note that listtasks is in fact a task itself, and that the -c option to bitbake allows you to explicitly run specific tasks. We'll make use of this in the next section when we discuss working with a recipe.