lueschem / edi-raspbian Goto Github PK

In general, the system works very well, and this isn't really a bug report; it's a request for comment on best practice.

Here's the issue:
I'm compiling a Raspbian-targeted application that depends on libudev-dev, which requires libudev1. This is a core library for Ubuntu and Raspbian (so lots of applications, like apt, break when this library gets screwed up). My understanding is that, for multiarch installations, the versions of each architecture must be identical. Here are the available versions as reported by sudo apt list libudev1:
libudev1/stable,now 241-7~deb10u5 amd64 [installed]
libudev1/stable 241-7~deb10u5+rpi1 armhf

So the versions are different by the "+rpi1".

Question:
What is the best practice for dealing with a situation like this? (If I just force the installation of the armhf version, it breaks the machine (no availability of apt; lots of other amd64 applications/libraries are removed.)

• Copy the relevant files (libudev1, libudev-dev include files, etc.) from a Raspbian machine into the corresponding armhf directories on the edi machine? (Lots of manual work and/or creating a script to do all the stuff that the package installer usually does.)
• Try to compile new version(s)--amd64 and armhf of the libraries, so I can do a make install? (This seems fraught and likely to result in similar 'damage' to libudev1-dependent packages.)
• Some other way? (There must be a better way???)

How do you deal with this situation when library versions are substantially identical, but not exactly so? If you have a good answer, it might be worth a blog post.

How to reproduce:

1. edi -v lxc configure jessie-cross-test jessie-cross.yml.
2. Setup fails with the following message:
   TASK [development_tools : Download corresponding armhf version of linux-libc-dev.] *** fatal: [jessie-cross-test]: FAILED! => {"changed": false, "dest": "/tmp/linux-libc-dev_3.16.7-ckt25-2_armhf.deb", "failed": true, "msg": "Request failed", "response": "HTTP Error 404: Not Found", "state": "absent", "status_code": 404, "url": "http://deb.debian.org/debian/pool/main/l/linux//linux-libc-dev_3.16.7-ckt25-2_armhf.deb"}

Root cause:

linux-libc-dev from jessie-updates got updated and therefore the version 3.16.7-ckt25-2 is no longer available.

nathan@Dell-latitude-E6510:~/edi-raspbian$ sudo edi -v lxc configure raspbian-buster-cross buster-cross.yml
INFO:root:Load time dictionary:
edi_configuration_name: buster-cross
edi_configure_remote_target: false
edi_create_distributable_image: false
edi_current_display: '0'
edi_current_user_gid: 1000
edi_current_user_group_name: nathan
edi_current_user_host_home_directory: /home/nathan
edi_current_user_name: nathan
edi_current_user_ssh_pub_keys: []
edi_current_user_target_home_directory: /home/nathan
edi_current_user_uid: 1000
edi_edi_plugin_directory: /usr/lib/python3/dist-packages/edi/plugins
edi_edi_version: 1.9.4+u1804
edi_host_ftp_proxy: ''
edi_host_hostname: Dell-latitude-E6510
edi_host_http_proxy: ''
edi_host_https_proxy: ''
edi_host_no_proxy: ''
edi_host_socks_proxy: ''
edi_log_level: INFO
edi_lxd_version: 3.0.3
edi_project_directory: /home/nathan/edi-raspbian
edi_project_plugin_directory: /home/nathan/edi-raspbian/plugins
edi_work_directory: /home/nathan/edi-raspbian

INFO:root:Using base configuration file 'buster-cross.yml'
INFO:root:Using overlay configuration file 'configuration/overlay/buster-cross.global.yml'
INFO:root:Merged configuration:
bootstrap:
additional_packages:

• python
• sudo
• netbase
• net-tools
• iputils-ping
• ifupdown
• isc-dhcp-client
• resolvconf
• systemd
• systemd-sysv
• gnupg
• dumb-init
  architecture: amd64
  repository: deb http://deb.debian.org/debian buster main
  repository_key: https://ftp-master.debian.org/keys/archive-key-9.asc
  documentation_steps: {}
  general:
  edi_compression: gz
  edi_required_minimal_edi_version: 1.1.0
  parameters:
  gcc_bootstrap_environment: false
  lxc_profiles:
  010_lxc_networking:
  path: lxc_profiles/general/lxc_networking/default_interface.yml
  020_default_root_device:
  path: lxc_profiles/general/default_root_device/default_root_device.yml
  030_suppress_init:
  path: lxc_profiles/general/suppress_init/suppress_init.yml
  skip: true
  lxc_templates:
  010_etc_hostname:
  path: lxc_templates/debian/hostname/hostname.yml
  020_etc_hosts:
  path: lxc_templates/debian/hosts/hosts.yml
  playbooks:
  010_base_system:
  parameters:
  install_openssh_server: true
  path: playbooks/debian/base_system/main.yml
  040_development_user_facilities:
  path: playbooks/debian/development_user_facilities/main.yml
  050_development_setup:
  parameters:
  enable_cross_development: true
  path: playbooks/development_setup_buster/main.yml
  postprocessing_commands: {}
  qemu: {}
  shared_folders:
  edi_workspace:
  folder: edi-workspace
  mountpoint: edi-workspace

Going to bootstrap initial image - be patient.
INFO:root:Running command: ['debootstrap', '--arch=amd64', '--variant=minbase', '--include=python,sudo,netbase,net-tools,iputils-ping,ifupdown,isc-dhcp-client,resolvconf,systemd,systemd-sysv,gnupg,dumb-init', '--components=main', '--force-check-gpg', '--keyring=/home/nathan/edi-raspbian/tmpkmbxibdj/temp_keyring.gpg', 'buster', '/home/nathan/edi-raspbian/tmpkmbxibdj/rootfs', 'http://deb.debian.org/debian']
I: Retrieving InRelease
I: Checking Release signature
E: Release signed by unknown key (key id DCC9EFBF77E11517)
Error: Command '['debootstrap', '--arch=amd64', '--variant=minbase', '--include=python,sudo,netbase,net-tools,iputils-ping,ifupdown,isc-dhcp-client,resolvconf,systemd,systemd-sysv,gnupg,dumb-init', '--components=main', '--force-check-gpg', '--keyring=/home/nathan/edi-raspbian/tmpkmbxibdj/temp_keyring.gpg', 'buster', '/home/nathan/edi-raspbian/tmpkmbxibdj/rootfs', 'http://deb.debian.org/debian']' returned non-zero exit status 1.
For more information increase the log level.

The setup of a cross compilation container fails behind a proxy:

Environment:
ansible: 2.3.1.0
lxc: 2.15
edi: 0.6.1

$ sudo edi -v lxc configure raspbian-jessie-cross-3 jessie-cross.yml

...

TASK [multiarch_support : Update apt cache.] **********************************************************************************************
changed: [raspbian-jessie-cross-3]

TASK [development_tools : Install specific version of native linux-libc-dev.] *************************************************************
changed: [raspbian-jessie-cross-3]

TASK [development_tools : Download corresponding armhf version of linux-libc-dev.] ********************************************************
fatal: [raspbian-jessie-cross-3]: FAILED! => {"changed": false, "dest": "/tmp/linux-libc-dev_3.16.7-ckt25-2_armhf.deb", "failed": true, "msg": "Request failed: <urlopen error [Errno 101] Network is unreachable>", "state": "absent", "url": "http://ftp.ch.debian.org/debian/pool/main/l/linux//linux-libc-dev_3.16.7-ckt25-2_armhf.deb"}
	to retry, use: --limit @/home/lueschm1/workspace/edi-raspbian/plugins/playbooks/development_setup/main.retry

PLAY RECAP ********************************************************************************************************************************
raspbian-jessie-cross-3    : ok=11   changed=10   unreachable=0    failed=1   

Error: Command '['sudo', '-u', 'lueschm1', 'ansible-playbook', '--connection', 'lxd', '--inventory', '/home/lueschm1/workspace/edi-raspbian/tmpiadj8xfl/inventory', '--extra-vars', '@/home/lueschm1/workspace/edi-raspbian/tmpiadj8xfl/extra_vars_050_development_setup', '/home/lueschm1/workspace/edi-raspbian/plugins/playbooks/development_setup/main.yml']' returned non-zero exit status 2
For more information increase the log level.

TASK [development_tools : Install cross development tools.] ********************
failed: [raspbian-jessie-cross] (item=[u'crossbuild-essential-armhf', u'dpkg-cross']) => {"cache_update_time": 0, "cache_updated": false, "failed": true, "item": ["crossbuild-essential-armhf", "dpkg-cross"], "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\"     install 'crossbuild-essential-armhf' 'dpkg-cross'' failed: E: Unable to correct problems, you have held broken packages.\n", "stderr": "E: Unable to correct problems, you have held broken packages.\n", "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSome packages could not be installed. This may mean that you have\nrequested an impossible situation or if you are using the unstable\ndistribution that some required packages have not yet been created\nor been moved out of Incoming.\nThe following information may help to resolve the situation:\n\nThe following packages have unmet dependencies:\n crossbuild-essential-armhf : Depends: gcc-arm-linux-gnueabihf (>= 4.9.1-1) but it is not going to be installed\n                              Depends: g++-arm-linux-gnueabihf (>= 4.9.1-1) but it is not going to be installed\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Some packages could not be installed. This may mean that you have", "requested an impossible situation or if you are using the unstable", "distribution that some required packages have not yet been created", "or been moved out of Incoming.", "The following information may help to resolve the situation:", "", "The following packages have unmet dependencies:", " crossbuild-essential-armhf : Depends: gcc-arm-linux-gnueabihf (>= 4.9.1-1) but it is not going to be installed", "                              Depends: g++-arm-linux-gnueabihf (>= 4.9.1-1) but it is not going to be installed"]}

NO MORE HOSTS LEFT *************************************************************
	to retry, use: --limit @/home/lueschem/workspace/edi-raspbian/plugins/playbooks/development_setup/main.retry

PLAY RECAP *********************************************************************
raspbian-jessie-cross      : ok=16   changed=11   unreachable=0    failed=1   

Error: Command '['sudo', '-u', 'lueschem', 'ansible-playbook', '--connection', 'lxd', '--inventory', '/home/lueschem/workspace/edi-raspbian/tmprulrnem2/inventory', '--extra-vars', '@/home/lueschem/workspace/edi-raspbian/tmprulrnem2/extra_vars_050_development_setup', '/home/lueschem/workspace/edi-raspbian/plugins/playbooks/development_setup/main.yml']' returned non-zero exit status 1
For more information increase the log level.

Reason: emdebian did not rebuild gcc after the retpoline updates. Therefore we end up with an incompatible version mix between the packages coming from emdebian, Debian, Debian security and Raspbian.

I build the debian buster, do you have a script for building the cross-compilers?

TASK [development_tools : Install corresponding versions of linux-libc-dev.] ***
failed: [raspbian-stretch-cross] (item=[u'linux-libc-dev', u'linux-libc-dev:armhf']) => {"cache_update_time": 0, "cache_updated": false, "failed": true, "item": ["linux-libc-dev", "linux-libc-dev:armhf"], "msg": "'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" install 'linux-libc-dev' 'linux-libc-dev:armhf'' failed: E: Unable to correct problems, you have held broken packages.\n", "stderr": "E: Unable to correct problems, you have held broken packages.\n", "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSome packages could not be installed. This may mean that you have\nrequested an impossible situation or if you are using the unstable\ndistribution that some required packages have not yet been created\nor been moved out of Incoming.\nThe following information may help to resolve the situation:\n\nThe following packages have unmet dependencies:\n linux-libc-dev:armhf : Breaks: linux-libc-dev (!= 4.9.82-1+deb9u3+rpi1) but 4.9.82-1+deb9u3 is to be installed\n linux-libc-dev : Breaks: linux-libc-dev:armhf (!= 4.9.82-1+deb9u3) but 4.9.82-1+deb9u3+rpi1 is to be installed\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Some packages could not be installed. This may mean that you have", "requested an impossible situation or if you are using the unstable", "distribution that some required packages have not yet been created", "or been moved out of Incoming.", "The following information may help to resolve the situation:", "", "The following packages have unmet dependencies:", " linux-libc-dev:armhf : Breaks: linux-libc-dev (!= 4.9.82-1+deb9u3+rpi1) but 4.9.82-1+deb9u3 is to be installed", " linux-libc-dev : Breaks: linux-libc-dev:armhf (!= 4.9.82-1+deb9u3) but 4.9.82-1+deb9u3+rpi1 is to be installed"]}

Hello, I tried to use edi-raspbian but the set up fails when ansible asks the raspbian-buster-cross to update and upgrade apt. Here the message. Do you know if something else is needed for this to work? I follow the instructions and have tried several times:

After trying:
sudo edi -v lxc configure raspbian-buster-cross buster-cross.yml

at one point I get this error:

TASK [apt_setup : Update and upgrade apt.] *************************************
[WARNING]: Updating cache and auto-installing missing dependency: python-apt
fatal: [raspbian-buster-cross]: FAILED! => {"changed": false, "cmd": "apt-get install --no-install-recommends python-apt -y -q", "msg": "E: Package 'python-apt' has no installation candidate", "rc": 100, "stderr": "E: Package 'python-apt' has no installation candidate\n", "stderr_lines": ["E: Package 'python-apt' has no installation candidate"], "stdout": "Reading package lists...\nBuilding dependency tree...\nPackage python-apt is not available, but is referred to by another package.\nThis may mean that the package is missing, has been obsoleted, or\nis only available from another source\n\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Package python-apt is not available, but is referred to by another package.", "This may mean that the package is missing, has been obsoleted, or", "is only available from another source", ""]}

Thanks!