E.g., bitbake multiconfig:qemuarm-wheezy:isar-image-base multiconfig:qemuarm-jessie:isar-image-base fails in isar-image-base.bb with the following error:

cp: cannot stat ‘/home/ibr/w/work/j/isar/src/b-master-1/tmp/work/isar-image-base/qemuarm/rootfs/boot/vmlinuz-3.2.0-4-vexpress’: No such file or directory

tmp/work/isar-image-base/qemuarm is used for both wheezy and jessie.

libhello build failed:

$ bitbake multiconfig:qemuarm-stretch:isar-image-base  multiconfig:de0-nano-soc-stretch:isar-image-base

mc:de0-nano-soc-stretch:libhello-0.1-98f2e41-r0 do_build:

| configure: creating ./config.status
|  /bin/bash ./config.status
| config.status: creating Makefile
| config.status: creating config.h
| config.status: executing depfiles commands
| config.status: executing libtool commands
| make  all-am
| make[2]: Entering directory '/home/builder/libhello/libhello-0.1-98f2e41'
| /bin/bash ./libtool  --tag=CC   --mode=compile arm-linux-gnueabihf-gcc -DHAVE_CONFIG_H    -Wdate-time -D_FORTIFY_SOURCE=2  -g -O2 -fdebug-prefix-map=/home/builder/libhello/libhello-0.1-98f2e41=. -fstack-protector-strong -Wformat -Werror=format-security -c -o hello.lo hello.c
| libtool: compile:  arm-linux-gnueabihf-gcc -DHAVE_CONFIG_H -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/home/builder/libhello/libhello-0.1-98f2e41=. -fstack-protector-strong -Wformat -Werror=format-security -c hello.c  -fPIC -DPIC -o .libs/hello.o
| libtool: compile:  arm-linux-gnueabihf-gcc -DHAVE_CONFIG_H -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/home/builder/libhello/libhello-0.1-98f2e41=. -fstack-protector-strong -Wformat -Werror=format-security -c hello.c -o hello.o >/dev/null 2>&1
| /bin/bash ./libtool  --tag=CC   --mode=link arm-linux-gnueabihf-gcc  -g -O2 -fdebug-prefix-map=/home/builder/libhello/libhello-0.1-98f2e41=. -fstack-protector-strong -Wformat -Werror=format-security  -Wl,-z,relro -o libhello.la -rpath /usr/lib/arm-linux-gnueabihf hello.lo
| libtool: link: `hello.lo' is not a valid libtool object
| Makefile:430: recipe for target 'libhello.la' failed
| make[2]: *** [libhello.la] Error 1
| make[2]: Leaving directory '/home/builder/libhello/libhello-0.1-98f2e41'
| Makefile:340: recipe for target 'all' failed
| make[1]: *** [all] Error 2
| make[1]: Leaving directory '/home/builder/libhello/libhello-0.1-98f2e41'
| dh_auto_build: make -j1 returned exit code 2
| debian/rules:22: recipe for target 'build' failed
| make: *** [build] Error 2
| dpkg-buildpackage: error: debian/rules build gave error exit status 2
| WARNING: exit code 2 from a shell command.
| ERROR: Function failed: do_build (log file is located at isar/build/tmp/work/debian-stretch-armhf/libhello-0.1-98f2e41-r0/temp/log.do_build.206

Specifying the environment variable DL_DIR doesn't work as expected, because ISAR default bitbake configuration overwrites it. DL_DIR = " is being used instead of DL_DIR ?= ".

asmirnov/next 6f689d7

console.txt
log.do_ext4_image.1276.txt

The wic tool calls mkfs.ext4 with option "-d". This option does not exist in the Debian jessie version. So it should be mentioned in the readme file that at least Debian stretch is required.

Build fails silently without qemu-arm-static. Please consider building without it or add an according check and at least a message in meta-isar/recipes-core/images/files/setup.sh and meta/recipes-devtools/buildchroot/files/setup.sh to make this failure reason easier to find.

Include option

omitdebsrc=true

into files

meta-isar/recipes-core/images/files/multistrap.conf.in
meta/recipes-devtools/buildchroot/files/multistrap.conf.in

to avoid import of Debian source repos.

Background: I want to use a company internal Debian mirror. But we do not mirror the source repository part of it.

root@sertac-build:/DATA/ISAR-Test2# git clone https://github.com/ilbers/isar.git
Cloning into 'isar'...
remote: Counting objects: 4069, done.
remote: Compressing objects: 100% (27/27), done.
remote: Total 4069 (delta 27), reused 39 (delta 22), pack-reused 4020
Receiving objects: 100% (4069/4069), 18.03 MiB | 5.31 MiB/s, done.
Resolving deltas: 100% (2124/2124), done.
root@sertac-build:/DATA/ISAR-Test2# ^C

root@sertac-build:/DATA/ISAR-Test2# cd isar/

root@sertac-build:/DATA/ISAR-Test2/isar# . isar-init-build-env ../build
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.

You had no conf/bblayers.conf file. This configuration file has therefore been
created for you with some default values. To add additional metadata layers
into your configuration please add entries to conf/bblayers.conf.

For more information about isar, contact the developers at:
https://lists.debian.org/debian-embedded/

Shell environment set up for builds.

You can now run 'bitbake '

Common targets are:
isar-image-base
isar-image-debug
root@sertac-build:/DATA/ISAR-Test2/build# bitbake -s
Traceback (most recent call last):
File "/DATA/ISAR-Test2/isar/bitbake/bin/bitbake", line 31, in
import bb
File "/DATA/ISAR-Test2/isar/bitbake/lib/bb/init.py", line 79, in
from bb import fetch2 as fetch
File "/DATA/ISAR-Test2/isar/bitbake/lib/bb/fetch2/init.py", line 1846, in
from . import clearcase
File "/DATA/ISAR-Test2/isar/bitbake/lib/bb/fetch2/clearcase.py", line 72, in
from distutils import spawn
ImportError: cannot import name 'spawn'
root@sertac-build:/DATA/ISAR-Test2/build# bitbake isar-image-base
Traceback (most recent call last):
File "/DATA/ISAR-Test2/isar/bitbake/bin/bitbake", line 31, in
import bb
File "/DATA/ISAR-Test2/isar/bitbake/lib/bb/init.py", line 79, in
from bb import fetch2 as fetch
File "/DATA/ISAR-Test2/isar/bitbake/lib/bb/fetch2/init.py", line 1846, in
from . import clearcase
File "/DATA/ISAR-Test2/isar/bitbake/lib/bb/fetch2/clearcase.py", line 72, in
from distutils import spawn
ImportError: cannot import name 'spawn'
root@sertac-build:/DATA/ISAR-Test2/build# ^C
root@sertac-build:/DATA/ISAR-Test2/build#

Date: Thu, 29 Jun 2017 17:52:24 +0200
From: Jan Kiszka
To: isar-users [email protected]

...because users do not have /sbin in their PATH. Where to fix this best? I don't want to fork imported wic files.

To: [email protected]
From: Jan Kiszka
Subject: Jessie: Host configuration leakage around apt-get
Date: Tue, 27 Jun 2017 19:38:46 +0200

Hi all,

for days, I was hunting a nasty bug around multistrap mystically failing in a jessie docker container while working fine in a stretch container as well as a regular jessie VM installation. It turned out that the docker image (debian:jessie-slim) contains

/etc/apt/apt.conf.d/docker-gzip-indexes

which disables the unpacking of packages index files after downloading. Due to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=774698, this also affected the multistrap call of apt-get. So, everyone trying to build Isar images with jessie can run into this bug. Maybe we should
detect the affect version (<2.2.1) and warn at least?

Considering to pull in 2.2.9 from stretch into my builder image now...

Jan

On 1/2/19 9:07 PM, Jan Kiszka wrote:

Hi all,

just saw this in CI build:

[...]
2019-01-02 19:37:55 - INFO - | localepurge: Removing disabled auto-generated config file.
2019-01-02 19:37:55 - INFO - |
2019-01-02 19:37:55 - INFO - | To reinstall all the packages which localepurge has been taking care
2019-01-02 19:37:55 - INFO - | of before, you can use the following command:
2019-01-02 19:37:55 - INFO - |
2019-01-02 19:37:55 - INFO - | apt-get -u --reinstall --fix-missing install $(dpkg -S LC_MESSAGES | cut -d: -f1 | tr ', ' '' | sort -u)
2019-01-02 19:37:55 - INFO - |
2019-01-02 19:37:55 - INFO - | mknod: /dev/console: File exists
2019-01-02 19:37:55 - INFO - | WARNING: exit code 1 from a shell command.
2019-01-02 19:37:55 - INFO - | ERROR: Function failed: do_rootfs (log file is located at /builds/ebsy/debian/build/tmp/work/jailhouse-demo-amd64/demo-image-qemu-amd64/temp/log.do_rootfs.66135)

I strongly suspect it comes from debian-configscript.sh:

Create console device if [ ! -e /dev/console ]; then mknod /dev/console c 5 1 fi
Something must be creating /dev/console in the background while the
config script runs. That is weird. Any ideas what could be the race
partner?

Do we still need this mknod? It date back to e7e5915. Removing this
would be the safest option because the case might be hard to reproduce -
I've only seen this once so far.

Jan

The project is missing contact information to reach out to in case of problems or suggestions. An issue tracker cannot replace a mailing list or something similar. However, even the github home isn't referenced.

How about creating a list? Could be on googlegroups if no mailman instance is available.

Provide such functionality

Currently multiconfigs in meta-isar contain more than just machine (MACHINE) and distribution (DISTRO) settings.

That sets a bad example, since bitbake and isar cannot distinguish between other possible settings that could have effected the build.

To fix this, all other configuration options should move into distro or machine configuration files.

Instead use http/https protocol instead

Build every package in its individual buildchroot to prevent dependency problems.

Use cases: TODO

Error log:

==================================================================
ERROR: hello-1.0-r0 do_unpack: Build of do_unpack failed
ERROR: hello-1.0-r0 do_unpack: Traceback (most recent call last):
File "/home/pfefferl/projects/isar/bitbake/lib/bb/build.py", line 642, in exec_task
return _exec_task(fn, task, d, quieterr)
File "/home/pfefferl/projects/isar/bitbake/lib/bb/build.py", line 580, in _exec_task
exec_func(task, localdata)
File "/home/pfefferl/projects/isar/bitbake/lib/bb/build.py", line 210, in exec_func
bb.utils.mkdirhier(adir)
File "/home/pfefferl/projects/isar/bitbake/lib/bb/utils.py", line 740, in mkdirhier
raise e
File "/home/pfefferl/projects/isar/bitbake/lib/bb/utils.py", line 737, in mkdirhier
os.makedirs(directory)
File "/usr/lib/python3.4/os.py", line 227, in makedirs
makedirs(head, mode, exist_ok)
File "/usr/lib/python3.4/os.py", line 237, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/home/pfefferl/projects/isar/build/tmp/work/buildchroot/debian-stretch/rootfs//home/builder'

ERROR: Task (multiconfig:qemu-amd64:/home/pfefferl/projects/isar/meta-isar/recipes-app/hello/hello.bb:do_unpack) failed with exit code '1'
NOTE: Tasks Summary: Attempted 4 tasks of which 3 didn't need to be rerun and 1 failed.

The mentions directory "..../home/builder" does really not exist!!

. isar-init-build-env ../b
bitbake isar-image-base
start_vm -d jessie

The above produces a working image, but fails due to:

/home/ibr/w/work/j/isar/src/b/tmp/deploy/images/vmlinuz-3.16.0-4-armmp: No such file or directory
qemu: could not load kernel '/home/ibr/w/work/j/isar/src/b/tmp/deploy/images/vmlinuz-3.16.0-4-armmp'

I saw it in the project README. Should start_armhf_vm be start_vm?

This issue could be reproduce by building target: multiconfig:qemuarm-wheezy:isar-image-base.
After build is complete, the following folder stays mounted:
tmp/work/debian-wheezy-armhf/buildchroot/rootfs/run/shm.

Isar creates root filesystem directories on the host with root privileges. Those are also required for tools like multistrap, mount and chroot. Currently, Isar requires using sudo ALL=NOPASSWD: ALL for simplicity.

The goal is to build without root privileges. Motivation:

• Improve security: Least privilege.
• Enable building on managed corporate hosts / containers.

Candidate tools:

• fakeroot: https://wiki.debian.org/Multistrap/Environment#fakeroot
• libpseudo:
  • https://www.yoctoproject.org/tools-resources/projects/pseudo
  • http://git.yoctoproject.org/cgit/cgit.cgi/pseudo/

See also:

• https://groups.google.com/forum/#!topic/isar-users/l-y_uwTXYSw

The kernel package version has changed in the debian repository. So the build fails now.

Better to use package "linux-image-amd64" instead of versioned package. The same is with vmlinuz and initrd. Better use /vmlinuz and /initrd.img instead of versioned names from /boot directory.

Provide such functionaliity

The user should be able to retrieve a list of packages and their exact versions that Isar installed in an image. With that list, it should then be possible to regenerate the very same image later on.

This also means we will need repository servers that preserver all those packages, even if normal mirrors dropped them with the availability of newer versions.

Provide such functionality

Building multiconfig:qemuarm-wheezy:isar-image-base and multiconfig:rpi-jessie:isar-image-base in a single bitbake call takes 12m7s, the same in two different build dirs and two bitbake instances takes 8m57s.

In addition, http://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html 2.4 says:

"Support for multiple configurations in this current release of the Yocto Project (Tbd 2.4) has some known issues:

• No inter-multi-configuration dependencies exist.

• Shared State (sstate) optimizations do not exist. Consequently, if the build uses the same object twice in, for example, two different TMPDIR directories, the build will either load from an existing sstate cache at the start or build the object twice."

What I noticed is that I cannot overwrite DISTRO_APT_SOURCE in local.conf with a local mirror because of

DISTRO_APT_SOURCE = "..."

Any reason why we should not make them all

DISTRO_APT_SOURCE ??= "..."

? Or is the plan rather to allow setting a set of mirrors and let the machinery pull from them when available, without local.conf overwrites?

Specifically for large packages like the kernel during development cycle, it would be more efficient to allow building those packages from source via the Debian cross-toolchain. Should be selectable in the recipe, maybe also globally via local.conf.

What is required to allow this? Any concrete ideas? We may have someone here who could contribute to this.

See 376b2f0 "meta/conf: Split bitbake config" and https://groups.google.com/d/msg/isar-users/kUN6lM03WpM/OXA0q6XBBQAJ.

It would be great, if isar (bitbake) is able to get the checksum for JForg Artifactory's using [1], automatically, not from "SRC_URI[sha256sum]" Or do we have a workaround to avoid hard coded checksum values in bb recipes.

[1] http://forums.jfrog.org/How-to-verify-curl-wget-download-artifact-is-correct-td7580081.html

See 376b2f0 "meta/conf: Split bitbake config" and https://groups.google.com/d/msg/isar-users/kUN6lM03WpM/OXA0q6XBBQAJ.

The project has 12 branches. And at one point @alexbluesman advised me to use asmirnov/next, though he has edited that to next

Question: Is it best to just always use next branch as using `master seems to be problematic?

Date: Thu, 29 Jun 2017 17:52:24 +0200
From: Jan Kiszka
To: isar-users [email protected]

I'm looking into IMAGE_TYPE = "wic" so that the user no longer has to call wic explicitly. We do this successfully in the yocto layer meta-iot2000 for quite a while.

Currently the main page readme says only Debian 8 is supported. May I know what are the plans for supporting Debian 9 if any?

As bitbake/lib/bb/build.py's shell_trap_code() injects set -e into scripts, the premature end of scripts leaves dangling mounts around. This may be fixed by hooking into the EXIT trap handler also setup by shell_trap_code(). See the patch and discussion on the Mailing List for details.

Currently, by the patches sent to the Mailing List, dangling mount handling is local to the scripts. Eventually, a more thorough approach may be taken, e.g., based on this proposal using an "unmount stack".

The docker image debian:stretch-slim is only 55 MB on x86 (granted, without kernel and modules), isar-image-base rootfs is 540 MB. It should be possible to provide a similarly slim image via Isar as well.

I'm quite new to using isar and bitbake but so far I found working with it a quite smooth experience.
However, I observed that the isar-setup-builddir script is only copying the content of bblayers.conf.sample and local.conf.sample to build/conf/.
See here

This will display an error message that the directory configured in .templateconf does not contain a local.conf file and also no bblayers.conf.
My naive assumption would be that I could provide a meta-isar/conf/local.conf right next to the meta-isar/conf/local.conf.sample and isar-setup-builddir would take this one instead and also not show the warning message.

Something like this:

if [ -r "$TEMPLATECONF/bblayers.conf"]; then
    ISARCORELAYERCONF="$TEMPLATECONF/bblayers.conf"
else
    ISARCORELAYERCONF="$TEMPLATECONF/bblayers.conf.sample"
fi

Or is this a design decision because of how Bitbake works? I myself like the approach of just deleting the whole build folder and then let isar take care of copying the files to the right locations.

Currently, the kernel version is hard-coded in the multiconfig file. Whenever Debian releases a new revision, the multiconfig file has to be updated. 52b8d35 determines the version automatically. It should be rebased onto the current master and updated to cover all multiconfigs.

Further use cases:

• Multiple versions in Debian.
• Additional kernels in product repos.
• Kernel not installed into the rootfs.
• No initrd.

I seem to be unable to get one due to #22 . Can I confirm if anyone else had success or is it just me?

Currently we don't have a clearly define pipeline of the tasks, especially for image recipes. This makes it difficult to introduce new tasks that should be run at certain points of the build process.

For image recipes I would like to propose the following pipeline:

1. do_rootfs a (virtual) task after which the rootfs directory contains the desired root file system with all packages installed, etc.
2. do_rootfs_postprocess a (virtual) task after which all post processing steps that can not be defined in a package are done.
3. do_image a virtual task after which the images are created in the work directory of the image.
4. do_deploy a (virtual) task after which the output artifacts of the recipe are deployed to the deploy directory.
5. do_build a virtual task that defines the default build process.

This way it should be simpler to order custom and currently implemented task better into the structure.
No more lines like these

addtask ubi_image before do_build after do_copy_boot_files do_install_imager_deps do_unpack do_transform_template

but instead

addtask ubi_image before do_image after do_rootfs_postprocess

I have not looked into optimizing the package build pipeline and other recipe types.

To reproduce:

• . isar-init-build-env ../build
• time bitbake multiconfig:qemuarm-stretch:isar-image-base
• start_vm

Boot log:
qemuarm-stretch-fail.txt

Problem description.

Following the user manual steps and running them in a Debian Stretch Virtualbox VM using Vagrant on Mac OS X 10.12.6, is unsuccessful

Expected / desired behavior.

Expecting a successful build

Actual behavior.

Build fails. Error msg

chroot: failed to run command ‘/configscript.sh’: Exec format error
WARNING: exit code 126 from a shell command.

How to reproduce.

• Install vagrant and virtualbox

vagrant box add debian/stretch64
vagrant init debian/stretch64
vagrant ssh
git clone http://github.com/ilbers/isar.git
cd isar/
. isar-init-build-env ../build
time bitbake isar-image-base isar-image-debug

We should start updating master like other projects do, ie. after qualifying next via reviews and CI to be probably stable. It's very misleading for observers that master is not normally maintained.