Would it be possible to create a meta-ptx scarthgap branch?

Hi,

I was updating our layers, and I ran into an issue with an incorrect filename passed to tar.

| NOTE: Unpacking /home/user/Documents/foo/build/tmp-glibc/deploy/images/nedap-foo/foo-base-nedap-foo.tar.bz2 to /home/user/Documents/foo/build/tmp-glibc/work/nedap_foo-oe-linux-gnueabi/foo-image/1.0/root
| tar: /home/user/Documents/foo/build/tmp-glibc/deploy/images/nedap-foo/foo-base-nedap-foo.tar.bz2: Cannot open: No such file or directory
| tar: Error is not recoverable: exiting now

I believe this is due to naming convention change in 4.3 https://docs.yoctoproject.org/singleindex.html#output-file-naming-changes .

The image is now named foo-base-nedap-foo.rootfs.tar.bz2 (recipe foo-base, machine nedap-foo). Note the addition of .rootfs.

I could work around this by setting

GENIMAGE_ROOTFS_IMAGE_FSTYPE = "rootfs.tar.bz2"

in my image recipe. However, I am not sure that this is an appropriate fix.

If there is more information needed, please let me know.

Thanks!

With meta-ptx master I am seeing the following error when using the genimage bbclass:

ERROR: could not open config file '[REDACTED]/.config'

, even though the path exists.

I think this is because the downstream patch was dropped when upgrading from confuse 3.2.2 to confuse 3.3, even though this fix isn't yet included in a confuse release.

As discussed in #121, dt-utils should be dropped from meta-ptx as it is redundant to dt-utils in meta-rauc. The recipes overlay each other. On top of that, the meta-ptx version defines debug packages. We should move them to the dt-utils recipe in meta-rauc and then delete dt-utils here.

Hi,

Do you think it could make sense to cherry-pick this fix 73c3252 on the dunfell branch as it is a long term support branch and this is a bug fix ?

Gilles

WARNING: simple-sd-image-1.0-r0 do_genimage: KeyError in ./deploy-simple-sd-image
ERROR: simple-sd-image-1.0-r0 do_genimage: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:sstate_report_unihash(d)
     0003:
File: '/home/phoenix/build/YOCTO.BSP-Pengutronix-OP-TEE/meta/classes/sstate.bbclass', lineno: 840, function: sstate_report_unihash
     0836:    report_unihash = getattr(bb.parse.siggen, 'report_unihash', None)
     0837:
     0838:    if report_unihash:
     0839:        ss = sstate_state_fromvars(d)
 *** 0840:        report_unihash(os.getcwd(), ss['task'], d)
     0841:}
     0842:
     0843:#
     0844:# Shell function to decompress and prepare a package for installation
File: '/home/phoenix/build/YOCTO.BSP-Pengutronix-OP-TEE/bitbake/lib/bb/siggen.py', lineno: 596, function: report_unihash
     0592:
     0593:            if "." in self.method:
     0594:                (module, method) = self.method.rsplit('.', 1)
     0595:                locs['method'] = getattr(importlib.import_module(module), method)
 *** 0596:                outhash = bb.utils.better_eval('method(path, sigfile, task, d)', locs)
     0597:            else:
     0598:                outhash = bb.utils.better_eval(self.method + '(path, sigfile, task, d)', locs)
     0599:
     0600:            try:
File: '/home/phoenix/build/YOCTO.BSP-Pengutronix-OP-TEE/bitbake/lib/bb/utils.py', lineno: 420, function: better_eval
     0416:    if extraglobals:
     0417:        ctx = copy.copy(ctx)
     0418:        for g in extraglobals:
     0419:            ctx[g] = extraglobals[g]
 *** 0420:    return eval(source, ctx, locals)
     0421:
     0422:@contextmanager
     0423:def fileslocked(files):
     0424:    """Context manager for locking and unlocking file locks."""
File: '<string>', lineno: 1, function: <module>
  File "<string>", line 1, in <module>

File: '/home/phoenix/build/YOCTO.BSP-Pengutronix-OP-TEE/meta/lib/oe/sstatesig.py', lineno: 592, function: OEOuthashBasic
     0588:
     0589:                update_hash("\n")
     0590:
     0591:            # Process this directory and all its child files
 *** 0592:            process(root)
     0593:            for f in files:
     0594:                if f == 'fixmepath':
     0595:                    continue
     0596:                process(os.path.join(root, f))
File: '/home/phoenix/build/YOCTO.BSP-Pengutronix-OP-TEE/meta/lib/oe/sstatesig.py', lineno: 555, function: process
     0551:                    add_perm(stat.S_IXOTH, 'x')
     0552:
     0553:                if include_owners:
     0554:                    try:
 *** 0555:                        update_hash(" %10s" % pwd.getpwuid(s.st_uid).pw_name)
     0556:                        update_hash(" %10s" % grp.getgrgid(s.st_gid).gr_name)
     0557:                    except KeyError:
     0558:                        bb.warn("KeyError in %s" % path)
     0559:                        raise
Exception: KeyError: 'getpwuid(): uid not found: 1000'

ERROR: Logfile of failure stored in: /home/phoenix/build/YOCTO.BSP-Pengutronix-OP-TEE/build/tmp/work/stm32dk2-oe-linux-gnueabi/simple-sd-image/1.0-r0/temp/log.do_genimage.2807
ERROR: Task (/home/phoenix/build/YOCTO.BSP-Pengutronix-OP-TEE/meta-ptx-optee/meta-ptx-optee-bsp/recipes-core/images/simple-sd-image.bb:do_genimage) failed with exit code '1'
NOTE: Tasks Summary: Attempted 2745 tasks of which 2744 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /home/phoenix/build/YOCTO.BSP-Pengutronix-OP-TEE/meta-ptx-optee/meta-ptx-optee-bsp/recipes-core/images/simple-sd-image.bb:do_genimage
Summary: There was 1 WARNING message shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.

This looks like a pseudo problem where pseudo tries to record the uid for the images contained in DEPLOYDIR, but those are already owned by the user and pseudo can't lookup the uid.

In file meta-ptx/classes/bootspec.bbclass in Line 33 the Variable BOOTSPEC_OPTIONS[doc] is set and in Line 36 is an set again. Should it be an append ?

Hello,

I found a compilation inconsistency in the way genimage.bbclass uses the shared state for the deploy task.
The genimage class inherits deploy. This allows reusing previously built images if the recipe's hashes haven't changed. The genimage.bbclass documentation meanwhile also recommends to add a dependency to the underlying rootfs image recipe in the form:

do_genimage[depends] += "core-image-minimal:do_image_complete"

However, image recipes like core-image-minimal do not inherit deploy.bbclass. This means they don't generate and reuse sstate artifacts. Hence, if I compile my genimage image in a new build directory with a populated sstate cache that matches (which happens a lot in CI), a new rootfs for core-image-minimal is reassembled, but we use the genimage .img from the sstate. If the rootfs build steps are not reproducible, for instance if I want to have a built timestamp like bellow, then the timestamps present on the rootfs .tar.bz2 and inside the genimage .img do not match!

REPRODUCIBLE_TIMESTAMP_ROOTFS = ""

Steps to reproduce:

1. Add REPRODUCIBLE_TIMESTAMP_ROOTFS = "" in the rootfs image recipe
2. Configure an external SSTATE_CACHE directory
3. Create a genimage recipe from that rootfs image recipe
4. Run bitbake to compile the genimage image
5. Remove the build directory, keep the SSTATE cache
6. Run bitbake to compile the genimage image again, while using the SSTATE cache
7. The /etc/version contained in the .tar.bz2 and .img do not match. This can also be seen easily from the symlink names in tmp/deploy

I noticed that other image bundling Yocto classes do not inherit the deploy class. It seems reserved for packages wich have a deploy output like kernels and bootloaders. For instance, image.bbclass or swupdate.bbclass do no inherit deploy. I question whether genimage should do it as well because of the bug explained above. I think we should directly write into the DEPLOY_DIR_IMAGE like those examples do. What is your opinion on this?

Thank you very much for your support!

Hi,
I'm not sure if I properly understood how to use the genimage.bbclass or how it's intended to work.

1. What's the recommended way to get the genimage.config from my recipe into the image source directory?
   since we're using the class in an image recipe, the fetch and unpack tasks are noexec, so SRC_URI won't work.
   I know there are ways to work around this, but I'd like to know how the authors intended it to work.

2. I understand that you extract the rootfs.tar.xz into GENIMAGE_ROOTDIR if a GENIMAGE_ROOTFS_IMAGE is given.

I have two problems with this:
2.1) I don't like the
do_genimage[depends] += "${@'${GENIMAGE_ROOTFS_IMAGE}:do_image_complete'
because I want to use the output in do_image_wic.
2.2) I want genimage to work on what's already in my IMAGE_ROOTFS after do_rootfs, so I'd like to skip the extraction part.

My solution was to simply not assign a GENIMAGE_ROOTFS_IMAGE.
But then the GENIMAGE_ROOTDIR is empty and genimage breaks.
Assigning GENIMAGE_ROOTDIR="${WORKDIR}/rootfs" also broke things.
So now I added another prefunc to just cp -a IMAGE_ROOTFS/* GENIMAGE_ROOTDIR
Is that the proper way to do it?

Thanks a lot for helping me understand!

OE-core has renamed the bmap-tools recipe to bmaptool, so now one gets

ERROR: Nothing PROVIDES 'bmap-tools-native'. Close matches:
  bmaptool-native
  bpftool-native
  mtools-native

They refuse to add the PROVIDES which would give compatibility

https://lore.kernel.org/openembedded-core/CANNYZj_5AkKa_dJVmjzvnng_7SFrdDr70T_wDN=bcA2WfMX+Bw@mail.gmail.com/
https://lore.kernel.org/openembedded-core/ba338ad83d02efbaeac55e4f80d9560cab92a54d.camel@linuxfoundation.org/

I don't really see any way to actually keep meta-ptx compatible with both nanbield and scarthgap (of course, this particular thing is only an issue when one sets GENIMAGE_CREATE_BMAP).