yocto 编译流程分析
git clone 一份poky 的工程到本地。
source poky/oe-init-build-env your_build_path
看下 oe-init-build-env 这个shell 脚本都干了些什么：
if [ -z "$ZSH_NAME" ] && [ "x$0" = "x./oe-init-build-env" ]; then
echo "Error: This script needs to be sourced. Please run as '. ./oe-init-build-env'"
else
if [ -n "$BASH_SOURCE" ]; then
OEROOT="`dirname $BASH_SOURCE`"
elif [ -n "$ZSH_NAME" ]; then
OEROOT="`dirname $0`"
else
OEROOT="`pwd`"
fi
OEROOT=`readlink -f "$OEROOT"`
export OEROOT
. $OEROOT/scripts/oe-buildenv-internal && \
$OEROOT/scripts/oe-setup-builddir && \
[ -n "$BUILDDIR" ] && cd $BUILDDIR
unset OEROOT
unset BBPATH
fi
转载： OSChina 上一个博友分析的这段shell 脚本：
http://my.oschina.net/u/158589/blog/70921
第一步，判断该脚本是否是用source或者.的方法运行的。 但是
1 |
["x$0"="x./oe-init-build-env"] |
只能对./oe-init-build-env这种执行方式报错，对./yocto/oe-init-build-env是不会报错的。
第二步，设置OEROOT这个变量。当用.或者source去执行脚本时，BASH_SOURCE这个变量会被自动设置到源文件路径。于是dirname $BASH_SOURCE就获得了脚本所在目录。readlink -f $OEROOT获得了绝对路径。
第三步，执行oe-buildenv-internal和oe-setup-builddir这两个脚本，并且进入到build目录。
（注意这里的oe-buildenv-internal是用.执行的，而oe-setup-build是fork shell执行的。因为oe-setup-builddir中用到了OEROOT这个变量，所以在此之前，OEROOT必须被export，正如脚本中所做的那样。）
第四步，unset一些变量。因为.或者source的执行方式是在原shell中执行，并不fork shell，所以如果不unset，会一直留在该shell中。
chenqi@chenqi-laptop ~/MyPro/ShellScript/yocto $ . ./oe-init-build-env
BASH_SOURCE = ./oe-init-build-env, OEROOT = .
OEROOT = /home/chenqi/MyPro/ShellScript/yocto
chenqi@chenqi-laptop ~/MyPro/ShellScript $ source yocto/oe-init-build-env
BASH_SOURCE = yocto/oe-init-build-env, OEROOT = yocto
OEROOT = /home/chenqi/MyPro/ShellScript/yocto
可见，无论在哪个目录下执行，最后获得的OEROOT的绝对路径都是一致的。（主要利用BASH_SOURCE, dirname, readlink）。
. $OEROOT/scripts/oe-buildenv-internaloe-buildenv-internal 这个shell 脚本主要是设置环境变量（poky/bitbake/bin/ 和 poky/scripts 这两个路径加入到PATH中。bitbake 命令就在poky/bitbake/bin/路径下面。），如下：
PATH="${OEROOT}/scripts:$BITBAKEDIR/bin/:$PATH"
unset BITBAKEDIR
# Used by the runqemu script
export BUILDDIR
export PATH
export BB_ENV_EXTRAWHITE="MACHINE DISTRO TCMODE TCLIBC HTTP_PROXY http_proxy \
HTTPS_PROXY https_proxy FTP_PROXY ftp_proxy FTPS_PROXY ftps_proxy ALL_PROXY \
all_proxy NO_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY \
SDKMACHINE BB_NUMBER_THREADS BB_NO_NETWORK PARALLEL_MAKE GIT_PROXY_COMMAND \
SOCKS5_PASSWD SOCKS5_USER SCREENDIR STAMPS_DIR"
$OEROOT/scripts/oe-setup-builddiroe-setup-builddir 这个shell 脚本，创建编译目录，判断当前编译目录下面是否存在conf/local.conf 文件，如果不存在local.conf 配置文件的话，通过Template模板，sample 生成一个local.conf。
下面这段shell 脚本的意思是首先检查 meta-yocto 目录下面是否存在conf/ 如果存在的话就用meta-yocto/conf/ 下面的 local.conf.sample 和 bblayers.conf， 如果不存在的话就到meta/conf 下面去找 local.conf.sample 和 bblayer.conf。
TEMPLATECONF=${TEMPLATECONF:-meta-yocto/conf}
#
# $TEMPLATECONF can point to a directory for the template local.conf & bblayers.conf
#
if [ "x" != "x$TEMPLATECONF" ]; then
if ! (test -d "$TEMPLATECONF"); then
# Allow TEMPLATECONF=meta-xyz/conf as a shortcut
if [ -d "$OEROOT/$TEMPLATECONF" ]; then
TEMPLATECONF="$OEROOT/$TEMPLATECONF"
fi
if ! (test -d "$TEMPLATECONF"); then
echo >&2 "Error: '$TEMPLATECONF' must be a directory containing local.conf & bblayers.conf"
return
fi
fi
OECORELAYERCONF="$TEMPLATECONF/bblayers.conf.sample"
OECORELOCALCONF="$TEMPLATECONF/local.conf.sample"
OECORENOTESCONF="$TEMPLATECONF/conf-notes.txt"
fi
if [ "x" = "x$OECORELOCALCONF" ]; then
OECORELOCALCONF="$OEROOT/meta/conf/local.conf.sample"
fi
if ! (test -r "$BUILDDIR/conf/local.conf"); then
cat <<EOM
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to use a
different MACHINE (target hardware) or enable parallel build options to take
advantage of multiple cores for example. See the file for more information as
common configuration options are commented.
The Yocto Project has extensive documentation about OE including a reference manual
which can be found at:
http://yoctoproject.org/documentation
For more information about OpenEmbedded see their website:
http://www.openembedded.org/
EOM
cp -f $OECORELOCALCONF $BUILDDIR/conf/local.conf
fi
if [ "x" = "x$OECORELAYERCONF" ]; then
OECORELAYERCONF="$OEROOT/meta/conf/bblayers.conf.sample"
fi
if ! (test -r "$BUILDDIR/conf/bblayers.conf"); then
cat <<EOM
You had no conf/bblayers.conf file. The configuration file has been created for
you with some default values. To add additional metadata layers into your
configuration please add entries to this file.
The Yocto Project has extensive documentation about OE including a reference manual
which can be found at:
http://yoctoproject.org/documentation
For more information about OpenEmbedded see their website:
http://www.openembedded.org/
EOM
# Put the abosolute path to the layers in bblayers.conf so we can run
# bitbake without the init script after the first run
sed "s|##COREBASE##|$OEROOT|g" $OECORELAYERCONF > $BUILDDIR/conf/bblayers.conf
fi
至此， build_path 下面的conf/local.conf 以及 bblayer.conf 文件就 创建好了，接下来就可以执行 bitbake target 编译了。
---------------------------------------------------------------- 华丽丽的分割线 ---------------------------------------------------------
现在开始分析bitbake 编译流程：
bitbake --help 可以看到bitbake 后面可以跟的选项参数：
yocto_build$ bitbake --help
Usage: bitbake [options] [package ...]
Executes the specified task (default is 'build') for a given set of BitBake files.
It expects that BBFILES is defined, which is a space separated list of files to
be executed. BBFILES does support wildcards.
Default BBFILES are the .bb files in the current directory.
Options:
--version show program's version number and exit
-h, --help show this help message and exit
-b BUILDFILE, --buildfile=BUILDFILE
execute the task against this .bb file, rather than a
package from BBFILES. Does not handle any
dependencies.
-k, --continue continue as much as possible after an error. While the
target that failed, and those that depend on it,
cannot be remade, the other dependencies of these
targets can be processed all the same.
-a, --tryaltconfigs continue with builds by trying to use alternative
providers where possible.
-f, --force force run of specified cmd, regardless of stamp status
-c CMD, --cmd=CMD Specify task to execute. Note that this only executes
the specified task for the providee and the packages
it depends on, i.e. 'compile' does not implicitly call
stage for the dependencies (IOW: use only if you know
what you are doing). Depending on the base.bbclass a
listtasks tasks is defined and will show available
tasks
-C INVALIDATE_STAMP, --clear-stamp=INVALIDATE_STAMP
Invalidate the stamp for the specified cmd such as
'compile' and run the default task for the specified
target(s)
-r PREFILE, --read=PREFILE
read the specified file before bitbake.conf
-R POSTFILE, --postread=POSTFILE
read the specified file after bitbake.conf
-v, --verbose output more chit-chat to the terminal
-D, --debug Increase the debug level. You can specify this more
than once.
-n, --dry-run don't execute, just go through the motions
-S, --dump-signatures
don't execute, just dump out the signature
construction information
-p, --parse-only quit after parsing the BB files (developers only)
-s, --show-versions show current and preferred versions of all recipes
-e, --environment show the global or per-package environment (this is
what used to be bbread)
-g, --graphviz emit the dependency trees of the specified packages in
the dot syntax, and the pn-buildlist to show the build
list
-I EXTRA_ASSUME_PROVIDED, --ignore-deps=EXTRA_ASSUME_PROVIDED
Assume these dependencies don't exist and are already
provided (equivalent to ASSUME_PROVIDED). Useful to
make dependency graphs more appealing
-l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS
Show debug logging for the specified logging domains
-P, --profile profile the command and print a report
-u UI, --ui=UI userinterface to use
-t SERVERTYPE, --servertype=SERVERTYPE
Choose which server to use, none, process or xmlrpc
--revisions-changed Set the exit code depending on whether upstream
floating revisions have changed or not
--server-only Run bitbake without UI, the frontend can connect with
bitbake server itself
-B BIND, --bind=BIND The name/address for the bitbake server to bind to
--no-setscene Do not run any setscene tasks, forces builds
那么，bitbake 对于这些个选项参数是怎么处理的呢。
bitbake target -xxx -xxxx -xx ，
这样其实就是执行poky/bitbake/bitbake 这个python 脚本:
这个python 脚本相对简单，可以看下最后的这段代码：
通过下面这个 if 判断，调用 main 函数执行，使用 python 库中的
optparse.OptionParser对选项参数进行处理。
if __name__ == "__main__":
try:
ret = main()#!/usr/bin/env python
# ex:ts=4:sw=4:sts=4:et
# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
import os
import sys, logging
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(__file__)),
'lib'))
import optparse
import warnings
from traceback import format_exception
try:
import bb
except RuntimeError as exc:
sys.exit(str(exc))
from bb import event
import bb.msg
from bb import cooker
from bb import ui
from bb import server
__version__ = "1.17.1"
logger = logging.getLogger("BitBake")
# Unbuffer stdout to avoid log truncation in the event
# of an unorderly exit as well as to provide timely
# updates to log files for use with tail
try:
if sys.stdout.name == '<stdout>':
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
except:
pass
class BBConfiguration(object):
"""
Manages build options and configurations for one run
"""
def __init__(self, options):
for key, val in options.__dict__.items():
setattr(self, key, val)
self.pkgs_to_build = []
*** ***
def main():
parser = optparse.OptionParser(
version = "BitBake Build Tool Core version %s, %%prog version %s" % (bb.__version__, __version__),
usage = """%prog [options] [package ...]
Executes the specified task (default is 'build') for a given set of BitBake files.
It expects that BBFILES is defined, which is a space separated list of files to
be executed. BBFILES does support wildcards.
Default BBFILES are the .bb files in the current directory.""")
parser.add_option("-b", "--buildfile", help = "execute the task against this .bb file, rather than a package from BBFILES. Does not handle any dependencies.",
action = "store", dest = "buildfile", default = None)
parser.add_option("-k", "--continue", help = "continue as much as possible after an error. While the target that failed, and those that depend on it, cannot be remade, the other dependencies of these targets can be processed all the same.",
action = "store_false", dest = "abort", default = True)
parser.add_option("-a", "--tryaltconfigs", help = "continue with builds by trying to use alternative providers where possible.",
action = "store_true", dest = "tryaltconfigs", default = False)
parser.add_option("-f", "--force", help = "force run of specified cmd, regardless of stamp status",
action = "store_true", dest = "force", default = False)
parser.add_option("-c", "--cmd", help = "Specify task to execute. Note that this only executes the specified task for the providee and the packages it depends on, i.e. 'compile' does not implicitly call stage for the dependencies (IOW: use only if you know what you are doing). Depending on the base.bbclass a listtasks tasks is defined and will show available tasks",
action = "store", dest = "cmd")
parser.add_option("-C", "--clear-stamp", help = "Invalidate the stamp for the specified cmd such as 'compile' and run the default task for the specified target(s)",
action = "store", dest = "invalidate_stamp")
parser.add_option("-r", "--read", help = "read the specified file before bitbake.conf",
action = "append", dest = "prefile", default = [])
parser.add_option("-R", "--postread", help = "read the specified file after bitbake.conf",
action = "append", dest = "postfile", default = [])
parser.add_option("-v", "--verbose", help = "output more chit-chat to the terminal",
action = "store_true", dest = "verbose", default = False)
parser.add_option("-D", "--debug", help = "Increase the debug level. You can specify this more than once.",
action = "count", dest="debug", default = 0)
parser.add_option("-n", "--dry-run", help = "don't execute, just go through the motions",
action = "store_true", dest = "dry_run", default = False)
parser.add_option("-S", "--dump-signatures", help = "don't execute, just dump out the signature construction information",
action = "store_true", dest = "dump_signatures", default = False)
parser.add_option("-p", "--parse-only", help = "quit after parsing the BB files (developers only)",
action = "store_true", dest = "parse_only", default = False)
parser.add_option("-s", "--show-versions", help = "show current and preferred versions of all recipes",
action = "store_true", dest = "show_versions", default = False)
parser.add_option("-e", "--environment", help = "show the global or per-package environment (this is what used to be bbread)",
action = "store_true", dest = "show_environment", default = False)
parser.add_option("-g", "--graphviz", help = "emit the dependency trees of the specified packages in the dot syntax, and the pn-buildlist to show the build list",
action = "store_true", dest = "dot_graph", default = False)
parser.add_option("-I", "--ignore-deps", help = """Assume these dependencies don't exist and are already provided (equivalent to ASSUME_PROVIDED). Useful to make dependency graphs more appealing""",
action = "append", dest = "extra_assume_provided", default = [])
parser.add_option("-l", "--log-domains", help = """Show debug logging for the specified logging domains""",
action = "append", dest = "debug_domains", default = [])
parser.add_option("-P", "--profile", help = "profile the command and print a report",
action = "store_true", dest = "profile", default = False)
parser.add_option("-u", "--ui", help = "userinterface to use",
action = "store", dest = "ui")
parser.add_option("-t", "--servertype", help = "Choose which server to use, none, process or xmlrpc",
action = "store", dest = "servertype")
parser.add_option("", "--revisions-changed", help = "Set the exit code depending on whether upstream floating revisions have changed or not",
action = "store_true", dest = "revisions_changed", default = False)
parser.add_option("", "--server-only", help = "Run bitbake without UI, the frontend can connect with bitbake server itself",
action = "store_true", dest = "server_only", default = False)
parser.add_option("-B", "--bind", help = "The name/address for the bitbake server to bind to",
action = "store", dest = "bind", default = False)
parser.add_option("", "--no-setscene", help = "Do not run any setscene tasks, forces builds",
action = "store_true", dest = "nosetscene", default = False)
options, args = parser.parse_args(sys.argv)
print "+++++ qc test +++++ options:", options
print "+++++ qc test +++++ args:", args
configuration = BBConfiguration(options)
configuration.pkgs_to_build.extend(args[1:])
print "+++++ qc test +++++ pkgs_to_build:",configuration.pkgs_to_build
*** ***
# Ensure logging messages get sent to the UI as events
handler = bb.event.LogHandler()
logger.addHandler(handler)
# Before we start modifying the environment we should take a pristine
# copy for possible later use
initialenv = os.environ.copy()
# Clear away any spurious environment variables while we stoke up the cooker
cleanedvars = bb.utils.clean_environment()
server = server.BitBakeServer()
if configuration.bind:
server.initServer((configuration.bind, 0))
else:
server.initServer()
idle = server.getServerIdleCB()
cooker = bb.cooker.BBCooker(configuration, idle, initialenv)
cooker.parseCommandLine()
server.addcooker(cooker)
server.saveConnectionDetails()
server.detach()
# Should no longer need to ever reference cooker
del cooker
logger.removeHandler(handler)
if not configuration.server_only:
# Setup a connection to the server (cooker)
server_connection = server.establishConnection()
# Restore the environment in case the UI needs it
for k in cleanedvars:
os.environ[k] = cleanedvars[k]
try:
return server.launchUI(ui_main, server_connection.connection, server_connection.events)
finally:
bb.event.ui_queue = []
server_connection.terminate()
else:
print("server address: %s, server port: %s" % (server.serverinfo.host, server.serverinfo.port))
return 1
if __name__ == "__main__":
try:
ret = main()
except Exception:
ret = 1
import traceback
traceback.print_exc(5)
sys.exit(ret)
这里调用到了 parseConfigurationFiles() 函数，这个函数用于解析 prefiles, postfiles (这两个变量都是空，需要通过执行bitbake -r -R 指定才行)，解析 layer.conf, bblayer.conf, bitbake.conf, 以及 bb 文件中 inherit 的 bbclass，以及包含base.bbclass。
def parseConfigurationFiles(self, prefiles, postfiles):
data = self.configuration.data
bb.parse.init_parser(data)
# Parse files for loading *before* bitbake.conf and any includes
for f in prefiles:
data = _parse(f, data)
layerconf = self._findLayerConf()
if layerconf:
parselog.debug(2, "Found bblayers.conf (%s)", layerconf)
data = _parse(layerconf, data)
layers = (data.getVar('BBLAYERS', True) or "").split()
data = bb.data.createCopy(data)
for layer in layers:
parselog.debug(2, "Adding layer %s", layer)
data.setVar('LAYERDIR', layer)
data = _parse(os.path.join(layer, "conf", "layer.conf"), data)
data.expandVarref('LAYERDIR')
data.delVar('LAYERDIR')
if not data.getVar("BBPATH", True):
raise SystemExit("The BBPATH variable is not set")
data = _parse(os.path.join("conf", "bitbake.conf"), data)
# Parse files for loading *after* bitbake.conf and any includes
for p in postfiles:
data = _parse(p, data)
# Handle any INHERITs and inherit the base class
bbclasses = ["base"] + (data.getVar('INHERIT', True) or "").split()
for bbclass in bbclasses:
data = _inherit(bbclass, data)
# Nomally we only register event handlers at the end of parsing .bb files
# We register any handlers we've found so far here...
for var in data.getVar('__BBHANDLERS') or []:
bb.event.register(var, data.getVar(var))
if data.getVar("BB_WORKERCONTEXT", False) is None:
bb.fetch.fetcher_init(data)
bb.codeparser.parser_cache_init(data)
bb.event.fire(bb.event.ConfigParsed(), data)
if data.getVar("BB_INVALIDCONF") is True:
data.setVar("BB_INVALIDCONF", False)
self.parseConfigurationFiles(self.configuration.prefile,
self.configuration.postfile)
else:
bb.parse.init_parser(data)
data.setVar('BBINCLUDED',bb.parse.get_file_depends(data))
self.configuration.data = data
self.configuration.data_hash = data.get_hash()
将所有的conf, bb, bbclass, etc 文件解析完毕之后，处理一下这些数据，提取出task list ，之后就是执行这些task 了。
下面以linux-yocto 为例看下如何找到这些task 的先后关系：
yocto_build/tmp/work/qemuppc-poky-linux/linux-yocto/3.4.36+gitAUTOINC+80b4b5110dca5184b57e85f1e775fb006a2e85ad_ddbc382cbc45a009e9ce17f7d448fcfd050ab5fc-r4.3/temp/log.task_order
这个文件显示了编译 linux-yocto 执行的task 以及 task 的执行顺序。
do_fetch (24333): log.do_fetch.24333
do_unpack (24359): log.do_unpack.24359
do_kernel_checkout (24371): log.do_kernel_checkout.24371
do_validate_branches (24399): log.do_validate_branches.24399
do_patch (24438): log.do_patch.24438
do_populate_lic (7751): log.do_populate_lic.7751
do_kernel_configme (7750): log.do_kernel_configme.7750
do_configure (18091): log.do_configure.18091
do_kernel_configcheck (18191): log.do_kernel_configcheck.18191
do_compile (23327): log.do_compile.23327
do_compile_kernelmodules (11394): log.do_compile_kernelmodules.11394
do_uboot_mkimage (11396): log.do_uboot_mkimage.11396
do_kernel_link_vmlinux (11397): log.do_kernel_link_vmlinux.11397
do_sizecheck (11395): log.do_sizecheck.11395
do_install (24128): log.do_install.24128
do_package (13631): log.do_package.13631
do_deploy (13632): log.do_deploy.13632
do_populate_sysroot (13633): log.do_populate_sysroot.13633
do_packagedata (16431): log.do_packagedata.16431
do_package_write_rpm (16452): log.do_package_write_rpm.16452
do_listtasks (7391): log.do_listtasks.7391
接下来介绍如何验证这些task 以及顺序：
poky/meta/recipes-kernel/linux/linux-yocto_3.4.bb
require recipes-kernel/linux/linux-yocto.inc
KBRANCH_DEFAULT = "standard/base"
KBRANCH = "${KBRANCH_DEFAULT}"
SRCREV_machine_qemuarm ?= "7cc80532306889b75619f8a1b713048e25f59e19"
SRCREV_machine_qemumips ?= "debce6677988e03b50c369aba5861d4f9b2e557d"
SRCREV_machine_qemuppc ?= "ddbc382cbc45a009e9ce17f7d448fcfd050ab5fc"
SRCREV_machine_qemux86 ?= "c994390cfa28339cbc1ec3b56eeec83a5fa75bb7"
SRCREV_machine_qemux86-64 ?= "c994390cfa28339cbc1ec3b56eeec83a5fa75bb7"
SRCREV_machine ?= "c994390cfa28339cbc1ec3b56eeec83a5fa75bb7"
SRCREV_meta ?= "80b4b5110dca5184b57e85f1e775fb006a2e85ad"
SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.4.git;protocol=git;bareclone=1;branch=${KBRANCH},${KMETA};name=machine,meta"
LINUX_VERSION ?= "3.4.36"
PR = "${INC_PR}.3"
PV = "${LINUX_VERSION}+git${SRCPV}"
KMETA = "meta"
COMPATIBLE_MACHINE = "qemuarm|qemux86|qemuppc|qemumips|qemux86-64"
# Functionality flags
KERNEL_FEATURES_append = " features/netfilter/netfilter.scc"
KERNEL_FEATURES_append_qemux86=" cfg/sound.scc"
KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc"
KERNEL_FEATURES_append_qemux86=" cfg/paravirt_kvm.scc"
KERNEL_FEATURES_append = " ${@bb.utils.contains("TUNE_FEATURES", "mx32", " cfg/x32.scc", "" ,d)}"
poky/meta/recipes-kernel/linux/linux-yocto.inc
DESCRIPTION = "Yocto Kernel"
SECTION = "kernel"
LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://COPYING;md5=d7810fab7487fb0aad327b76f1be7cd7"
INC_PR = "r4"
# A KMACHINE is the mapping of a yocto $MACHINE to what is built
# by the kernel. This is typically the branch that should be built,
# and it can be specific to the machine or shared
# KMACHINE = "UNDEFINED"
LINUX_KERNEL_TYPE ?= "standard"
# KMETA ?= ""
KBRANCH ?= "master"
KMACHINE ?= "${MACHINE}"
SRCREV_FORMAT ?= "meta_machine"
LINUX_VERSION_EXTENSION ?= "-yocto-${LINUX_KERNEL_TYPE}"
do_patch[depends] = "kern-tools-native:do_populate_sysroot"
addtask kernel_configme before do_configure after do_patch
# Pick up shared functions
inherit kernel
inherit kernel-yocto
require linux-dtb.inc
B = "${WORKDIR}/linux-${MACHINE}-${LINUX_KERNEL_TYPE}-build"
do_install_append(){
if [ -n "${KMETA}" ]; then
rm -rf ${STAGING_KERNEL_DIR}/${KMETA}
fi
}
# extra tasks
addtask kernel_link_vmlinux after do_compile before do_install
addtask validate_branches before do_patch after do_kernel_checkout
addtask kernel_configcheck after do_configure before do_compile
poky/meta/classes/base.bbclass
poky/meta/classes/kernel.bbclass
poky/meta/classes/kernel-yocto.bbclass
这是 linux-yocto 所有相关的bb, bbclass 文件，所有的task 都是通过addtasks 关键字添加的，它们之间先后关系构成了所谓的依赖关系，第一个task，在这几个bb, bbclass 文件里面肯定没有 addtask before the_1st_task 这样的语句。
poky/meta/classes$ grep -nr "addtask . | grep "patch"
就按照上面这样的方法不断的grep 就能验证出log.task_order 里面显示的task，以及 正确的执行顺序。
ex. $ bitbake linux-yocto -c cleanall 执行 cleanall task 需要运行的tasklist：
poky/meta/classes$ grep -nr "addtask" . | grep "clean"
./base.bbclass:636:addtask cleansstate after do_clean
./base.bbclass:637:addtask qc_test after do_cleansstate
./base.bbclass:648:addtask cleanall after do_cleansstate
./utility-tasks.bbclass:16:addtask clean
其中，qc_test task 是我自己加的测试 task:
addtask cleansstate after do_clean
addtask qc_test after do_cleansstate
do_qc_test() {
echo "qc hello base.bbclass !"
echo "qc test !!!!!~~~~ "
}
这样，构建起来的tasklist 如下：
| clean | clean |
| cleansstate | cleansstate |
| cleanall | qc_test |
如此，可以看出，执行 cleanall task 的话，clean, cleansstate task 都要执行，因为具有依赖关系，相反，qc_test 和 cleanall 虽然都是 after do_cleansstate ，但是二者之间没有依赖关系。其它的task 的 runqueue list 也是这样得到。
bitbake 如果不刻意指定要执行的task 的话，默认执行的是build 操作，而这个操作是针对一系列的bb 文件，这些文件是在BBFILES定义的。看一下BBFILES 这个变量的来历，方法如下
/poky/bitbake$ grep -nr "BBFILES" .
直接在bitbake 目录下面搜索有谁对BBFILES 这个变量进行了赋值操作，这样就能定位到./lib/bb/cooker.py
反向推理一下：
搜集bbfiles 函数：
def collect_bbfiles( self ):
"""Collect all available .bb build files"""
parsed, cached, skipped, masked = 0, 0, 0, 0
collectlog.debug(1, "collecting .bb files")
files = (data.getVar( "BBFILES", self.configuration.data, True) or "").split()
data.setVar("BBFILES", " ".join(files), self.configuration.data)
# Sort files by priority
files.sort( key=lambda fileitem: self.calc_bbfile_priority(fileitem) )
def matchFile(self, buildfile):
"""
Find the .bb file which matches the expression in 'buildfile'.
Raise an error if multiple files
"""
matches = self.matchFiles(buildfile)
def buildFile(self, buildfile, task):
"""
Build the file matching regexp buildfile
"""
# Too many people use -b because they think it's how you normally
# specify a target to be built, so show a warning
bb.warn("Buildfile specified, dependencies will not be handled. If this is not what you want, do not use -b / --buildfile.")
# Parse the configuration here. We need to do it explicitly here since
# buildFile() doesn't use the cache
self.parseConfiguration()
# If we are told to do the None task then query the default task
if (task == None):
task = self.configuration.cmd
fn, cls = bb.cache.Cache.virtualfn2realfn(buildfile)
fn = self.matchFile(fn)
self.buildSetVars()
def parseCommandLine(self):
# Parse any commandline into actions
self.commandlineAction = {'action':None, 'msg':None}
if self.configuration.show_environment:
if 'world' in self.configuration.pkgs_to_build:
self.commandlineAction['msg'] = "'world' is not a valid target for --environment."
elif 'universe' in self.configuration.pkgs_to_build:
self.commandlineAction['msg'] = "'universe' is not a valid target for --environment."
elif len(self.configuration.pkgs_to_build) > 1:
self.commandlineAction['msg'] = "Only one target can be used with the --environment option."
elif self.configuration.buildfile and len(self.configuration.pkgs_to_build) > 0:
self.commandlineAction['msg'] = "No target should be used with the --environment and --buildfile options."
elif len(self.configuration.pkgs_to_build) > 0:
self.commandlineAction['action'] = ["showEnvironmentTarget", self.configuration.pkgs_to_build]
self.configuration.data.setVar("BB_CONSOLELOG", None)
else:
self.commandlineAction['action'] = ["showEnvironment", self.configuration.buildfile]
self.configuration.data.setVar("BB_CONSOLELOG", None)
elif self.configuration.buildfile is not None:
self.commandlineAction['action'] = ["buildFile", self.configuration.buildfile, self.configuration.cmd]
elif self.configuration.revisions_changed:
self.commandlineAction['action'] = ["compareRevisions"]
elif self.configuration.show_versions:
self.commandlineAction['action'] = ["showVersions"]
elif self.configuration.parse_only:
self.commandlineAction['action'] = ["parseFiles"]
elif self.configuration.dot_graph:
if self.configuration.pkgs_to_build:
self.commandlineAction['action'] = ["generateDotGraph", self.configuration.pkgs_to_build, self.configuration.cmd]
else:
self.commandlineAction['msg'] = "Please specify a package name for dependency graph generation."
else:
if self.configuration.pkgs_to_build:
self.commandlineAction['action'] = ["buildTargets", self.configuration.pkgs_to_build, self.configuration.cmd]
else:
#self.commandlineAction['msg'] = "Nothing to do. Use 'bitbake world' to build everything, or run 'bitbake --help' for usage information."
self.commandlineAction = None仔细看下上面这个函数，是不是和最初在bitbake 这个python 根脚本中定义的main 函数有很多相似之处，这样就能猜到 bitbake -b xxx 这个指令回执行到buildFile 这个python 函数。同样可以知道 bitbake -e 可以看到所有的环境变量，包括BBFILES 变量的值，因为它执行了 showEnvrioment 这个python 函数。
强烈建议把 bitbake -e > ~/bitbake_-e.txt 重定向到文件中好好看看。
def main():
parser = optparse.OptionParser(
version = "BitBake Build Tool Core version %s, %%prog version %s" % (bb.__version__, __version__),
usage = """%prog [options] [package ...]
Executes the specified task (default is 'build') for a given set of BitBake files.
It expects that BBFILES is defined, which is a space separated list of files to
be executed. BBFILES does support wildcards.
Default BBFILES are the .bb files in the current directory.""")
parser.add_option("-b", "--buildfile", help = "execute the task against this .bb file, rather than a package from BBFILES. Does not handle any dependencies.",
action = "store", dest = "buildfile", default = None)
parser.add_option("-k", "--continue", help = "continue as much as possible after an error. While the target that failed, and those that depend on it, cannot be remade, the other dependencies of these targets can be processed all the same.",
action = "store_false", dest = "abort", default = True)
parser.add_option("-a", "--tryaltconfigs", help = "continue with builds by trying to use alternative providers where possible.",
action = "store_true", dest = "tryaltconfigs", default = False)
parser.add_option("-f", "--force", help = "force run of specified cmd, regardless of stamp status",
action = "store_true", dest = "force", default = False)
parser.add_option("-c", "--cmd", help = "Specify task to execute. Note that this only executes the specified task for the providee and the packages it depends on, i.e. 'compile' does not implicitly call stage for the dependencies (IOW: use only if you know what you are doing). Depending on the base.bbclass a listtasks tasks is defined and will show available tasks",
action = "store", dest = "cmd")
OK.
再看一下 parseCommandLine 这个函数if elif elif ,,, else 通过不断的探测bitbake 的选项参数，如果没有选项参数，只有target 走到buildTargets 这个else 里面，接下来要执行的函数是buildTargets.
def parseCommandLine(self):
# Parse any commandline into actions
self.commandlineAction = {'action':None, 'msg':None}
if self.configuration.show_environment:
if 'world' in self.configuration.pkgs_to_build:
self.commandlineAction['msg'] = "'world' is not a valid target for --environment."
elif 'universe' in self.configuration.pkgs_to_build:
self.commandlineAction['msg'] = "'universe' is not a valid target for --environment."
elif len(self.configuration.pkgs_to_build) > 1:
self.commandlineAction['msg'] = "Only one target can be used with the --environment option."
elif self.configuration.buildfile and len(self.configuration.pkgs_to_build) > 0:
self.commandlineAction['msg'] = "No target should be used with the --environment and --buildfile options."
elif len(self.configuration.pkgs_to_build) > 0:
self.commandlineAction['action'] = ["showEnvironmentTarget", self.configuration.pkgs_to_build]
self.configuration.data.setVar("BB_CONSOLELOG", None)
else:
self.commandlineAction['action'] = ["showEnvironment", self.configuration.buildfile]
self.configuration.data.setVar("BB_CONSOLELOG", None)
elif self.configuration.buildfile is not None:
self.commandlineAction['action'] = ["buildFile", self.configuration.buildfile, self.configuration.cmd]
elif self.configuration.revisions_changed:
self.commandlineAction['action'] = ["compareRevisions"]
elif self.configuration.show_versions:
self.commandlineAction['action'] = ["showVersions"]
elif self.configuration.parse_only:
self.commandlineAction['action'] = ["parseFiles"]
elif self.configuration.dot_graph:
if self.configuration.pkgs_to_build:
self.commandlineAction['action'] = ["generateDotGraph", self.configuration.pkgs_to_build, self.configuration.cmd]
else:
self.commandlineAction['msg'] = "Please specify a package name for dependency graph generation."
else:
if self.configuration.pkgs_to_build:
self.commandlineAction['action'] = ["buildTargets", self.configuration.pkgs_to_build, self.configuration.cmd]
else:
#self.commandlineAction['msg'] = "Nothing to do. Use 'bitbake world' to build everything, or run 'bitbake --help' for usage information."
self.commandlineAction = None
看一个特定的package 编译流程，拿kernel 看吧：
从bitbake_-e.txt 环境中搜到：
PREFERRED_PROVIDER_virtual/kernel="linux-yocto"
PREFERRED_VERSION_linux-yocto="3.4%"
这样就可以在poky 目录下面搜索 名字为 linux-yocto*.bb* 的 bb和bbappend， 搜出来以后再取 3.4版本的那些bb 和 bbappend。
/poky$ find -name "linux-yocto*.bb*"
./meta/recipes-kernel/linux/linux-yocto_3.8.bb
./meta/recipes-kernel/linux/linux-yocto-dev.bb
./meta/recipes-kernel/linux/linux-yocto-rt_3.4.bb
./meta/recipes-kernel/linux/linux-yocto-rt_3.8.bb
./meta/recipes-kernel/linux/linux-yocto-rt_3.2.bb
./meta/recipes-kernel/linux/linux-yocto-tiny_3.2.bb
./meta/recipes-kernel/linux/linux-yocto_3.4.bb
./meta/recipes-kernel/linux/linux-yocto_3.2.bb
./meta/recipes-kernel/linux/linux-yocto-tiny_3.4.bb
./meta/recipes-kernel/linux/linux-yocto-tiny_3.8.bb
./meta-yocto-bsp/recipes-kernel/linux/linux-yocto_3.8.bbappend
./meta-yocto-bsp/recipes-kernel/linux/linux-yocto_3.2.bbappend
./meta-yocto-bsp/recipes-kernel/linux/linux-yocto_3.4.bbappend
./meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb
查看./meta/recipes-kernel/linux/linux-yocto_3.4.bb
require recipes-kernel/linux/linux-yocto.inc
KBRANCH_DEFAULT = "standard/base"
KBRANCH = "${KBRANCH_DEFAULT}"
SRCREV_machine_qemuarm ?= "7cc80532306889b75619f8a1b713048e25f59e19"
SRCREV_machine_qemumips ?= "debce6677988e03b50c369aba5861d4f9b2e557d"
SRCREV_machine_qemuppc ?= "ddbc382cbc45a009e9ce17f7d448fcfd050ab5fc"
SRCREV_machine_qemux86 ?= "c994390cfa28339cbc1ec3b56eeec83a5fa75bb7"
SRCREV_machine_qemux86-64 ?= "c994390cfa28339cbc1ec3b56eeec83a5fa75bb7"
SRCREV_machine ?= "c994390cfa28339cbc1ec3b56eeec83a5fa75bb7"
SRCREV_meta ?= "80b4b5110dca5184b57e85f1e775fb006a2e85ad"
SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.4.git;protocol=git;bareclone=1;branch=${KBRANCH},${KMETA};name=machine,meta"
LINUX_VERSION ?= "3.4.36"
PR = "${INC_PR}.3"
PV = "${LINUX_VERSION}+git${SRCPV}"
KMETA = "meta"
COMPATIBLE_MACHINE = "qemuarm|qemux86|qemuppc|qemumips|qemux86-64"
# Functionality flags
KERNEL_FEATURES_append = " features/netfilter/netfilter.scc"
KERNEL_FEATURES_append_qemux86=" cfg/sound.scc"
KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc"
KERNEL_FEATURES_append_qemux86=" cfg/paravirt_kvm.scc"
KERNEL_FEATURES_append = " ${@bb.utils.contains("TUNE_FEATURES", "mx32", " cfg/x32.scc", "" ,d)}"
先查看下 linux-yocto_3.4.bb require 的 recipes-kernel/linux/linux-yocto.inc
DESCRIPTION = "Yocto Kernel"
SECTION = "kernel"
LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://COPYING;md5=d7810fab7487fb0aad327b76f1be7cd7"
INC_PR = "r4"
# A KMACHINE is the mapping of a yocto $MACHINE to what is built
# by the kernel. This is typically the branch that should be built,
# and it can be specific to the machine or shared
# KMACHINE = "UNDEFINED"
LINUX_KERNEL_TYPE ?= "standard"
# KMETA ?= ""
KBRANCH ?= "master"
KMACHINE ?= "${MACHINE}"
SRCREV_FORMAT ?= "meta_machine"
LINUX_VERSION_EXTENSION ?= "-yocto-${LINUX_KERNEL_TYPE}"
do_patch[depends] = "kern-tools-native:do_populate_sysroot"
addtask kernel_configme before do_configure after do_patch
# Pick up shared functions
inherit kernel
inherit kernel-yocto
require linux-dtb.inc
B = "${WORKDIR}/linux-${MACHINE}-${LINUX_KERNEL_TYPE}-build"
do_install_append(){
if [ -n "${KMETA}" ]; then
rm -rf ${STAGING_KERNEL_DIR}/${KMETA}
fi
}
# extra tasks
addtask kernel_link_vmlinux after do_compile before do_install
addtask validate_branches before do_patch after do_kernel_checkout
addtask kernel_configcheck after do_configure before do_compile这里inherit kernel, inherit kernel-yocto, 在meta/class 里面可以看到 kernel.class ，kernel-yocto.class 文件，里面有kernel 公共的base 函数。
http://blog.chinaunix.net/uid-7652108-id-2047290.html
bitbake parse 的机理分析：
膜拜这个大神，早在07年就把 bitbake 一个package 的流程分析的这么透彻了：
http://blog.chinaunix.net/uid-7652108-id-2047247.html
在 linux-yocto.inc 文件中 inherit kernel-yocto， 来看下 class/kernel-yocto.bbclass , 分析一下do_kernel_configme 这个函数。添加log语句打印一下这个函数中的一些变量。 ${S} 表示linux source code 的目录， ${B} 表示linux build 目录。
在 linux_source_code_path/meta/scripts/configme 执行这个shell 脚本。
do_kernel_configme[dirs] = "${S} ${B}"
do_kernel_configme() {
echo "[INFO] doing kernel configme"
export KMETA=${KMETA}
if [ -n ${KCONFIG_MODE} ]; then
configmeflags=${KCONFIG_MODE}
else
# If a defconfig was passed, use =n as the baseline, which is achieved
# via --allnoconfig
if [ -f ${WORKDIR}/defconfig ]; then
configmeflags="--allnoconfig"
fi
fi
cd ${S}
PATH=${PATH}:${S}/scripts/util
configme ${configmeflags} --reconfig --output ${B} ${LINUX_KERNEL_TYPE} ${KMACHINE}
if [ $? -ne 0 ]; then
echo "ERROR. Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
exit 1
fi
#qc added log
echo "qc variables value: ${S} ${B} ${KMETA} ${KCONFIG_MODE} ${WORKDIR} ${LINUX_KERNEL_TYPE} ${KMACHINE}."
echo "# Global settings from linux recipe" >> ${B}/.config
echo "CONFIG_LOCALVERSION="\"${LINUX_VERSION_EXTENSION}\" >> ${B}/.config
}
configme 这个 shell 脚本完成 生成.config 文件的工作：
# This is factored out into a function because for a given branch,
# there may be more than one user (i.e. big endian, little endian,
# or BSPs that use the same branch but differ only in kernel configs)
run_board_config()
{
# Can't set these until we've unwound the checkpoint and have meta data.
KVER=`cat ./$META_DIR/cfg/kernel-*cache/kver|sed 's/^v//'`
# Look for standard defines, with compatibility fallbacks
KARCH=`grep KARCH $SCC | awk '{print $3}'`
KPROFILE=`grep KMACHINE $SCC | awk '{print $3}'`
KTYPE=`grep KTYPE $SCC | awk '{print $3}'`
META=./$META_DIR/meta-series
META_ALT=./$META_DIR/cfg/scratch/`basename $SCC .scc`-meta
BUILD_DIR=$out_dir
CFGFILE=$machine-$target-config-$KVER
kgit-meta -v -k $META
if [ $? != 0 ]; then
echo Error running the meta series for collecting config data
return 1
fi
KTGT=`get_branch_name $META`
mkdir -p ./$META_DIR/cfg/$KTGT
if [ $? != 0 ]; then
echo Failed to mkdir ./$META_DIR/cfg/$KTGT for config data
return 1
fi
frags=`cat $META_DIR/cfg/$KTGT/config_frag.txt | sed 's%\(^.*$\)%'$META_DIR/cfg'\1%'`
pre_config -l $META_DIR/cfg/$KTGT/ $frags > $META_DIR/cfg/$KTGT/config.log 2>&1
# remove any old assembled debug fragments
rm -f $BUILD_DIR/.tmp.config*
merge_frags=`cat $META_DIR/cfg/$KTGT/config_frag.txt | sed 's%\(^.*$\)%'$META_DIR/cfg'\1.sanitized%'`
ARCH=$KARCH O=$BUILD_DIR merge_config.sh $allnoconfig -d $merge_frags \
> $META_DIR/cfg/$KTGT/merge_log.txt 2>&1
mv $BUILD_DIR/.tmp.config* $META_DIR/cfg/$KTGT/$CFGFILE
if [ $? != 0 ]; then
echo creation of pre-processed config data failed
return 1
fi
# break the merge log down into parts that can be processed later
grep -A2 "^Value of" $META_DIR/cfg/$KTGT/merge_log.txt > $META_DIR/cfg/$KTGT/redefinition.txt
grep -A2 "^Value requested" $META_DIR/cfg/$KTGT/merge_log.txt > $META_DIR/cfg/$KTGT/mismatch.txt
echo "[INFO] Pre-processed cfg file $CFGFILE created."
拼接一系列的meta 目录下的config_frag 文件，最终通过 merge_config.sh 生成 .config 文件。
/yocto_build/tmp/work/qemuppc-poky-linux/linux-yocto/3.4.36+gitAUTOINC+80b4b5110dca5184b57e85f1e775fb006a2e85ad_ddbc382cbc45a009e9ce17f7d448fcfd050ab5fc-r4.3/linux/meta/cfg/standard/qemuppc/config_frag.txt
这个就是要处理的所有 config 的片段。
可以参考 config.log 和 merge_log.txt
/yocto_build/tmp/work/qemuppc-poky-linux/linux-yocto/3.4.36+gitAUTOINC+80b4b5110dca5184b57e85f1e775fb006a2e85ad_ddbc382cbc45a009e9ce17f7d448fcfd050ab5fc-r4.3/linux/meta/cfg/standard/qemuppc/config.log
/yocto_build/tmp/work/qemuppc-poky-linux/linux-yocto/3.4.36+gitAUTOINC+80b4b5110dca5184b57e85f1e775fb006a2e85ad_ddbc382cbc45a009e9ce17f7d448fcfd050ab5fc-r4.3/linux/meta/cfg/standard/qemuppc/merge_log.txt
至此，就知道 yocto 编译 linux kernel 的 .config 是从哪来的了！！！
了解更多关于 yocto kernel config 可以 git clone git://git.yoctoproject.org/yocto-kernel-cache，查看 00-README 文档。
http://blog.csdn.net/fmddlmyy/article/details/325403
在 bitbake 中加入 print log语句，方便调试：
parser.add_option("-B", "--bind", help = "The name/address for the bitbake server to bind to",
action = "store", dest = "bind", default = False)
parser.add_option("", "--no-setscene", help = "Do not run any setscene tasks, forces builds",
action = "store_true", dest = "nosetscene", default = False)
options, args = parser.parse_args(sys.argv)
#qc test code
print "qc test code: options:", options
print "qc test code: args", args
configuration = BBConfiguration(options)
configuration.pkgs_to_build.extend(args[1:])