xolox / python-apt-mirror-updater Goto Github PK

Instead of using https://launchpad.net/ubuntu/+archivemirrors, one could use instead http://mirrors.ubuntu.com.

It features a list of mirrors per country, or gives a list of mirrors given your IP, if using http://mirrors.ubuntu.com/mirrors.txt

These lists are way shorter than the whole list at launchpad.net, and then are faster to test for the best mirror for example, given that a mirror in your country is most of the time closer to you than the others.

The only downside is that it is using http instead of https.

I'm using elementary OS 6 (Odin) based on Ubuntu 20.04 LTS and changed some of your code to support this distro:

• Added a release entry in /backends/elementary.py
• Added support for https and mirror protocols in ./init.py

It's working fine for me. I'll submit a pull request so you can analyze if there's impact on different distros.

python-apt-mirror-updater assumes that a release that is EOL needs to have its sources updated to the old-releases version. This is incorrect as the move to old-releases can be delayed by quite some time, especially for LTS releases apparently (and there seems to be no schedule set from Canonical).

I propose to add a verification that the repositories for the release are indeed available in old-releases before installing the old-releases sources. This would prevent breaking an otherwise functional system.

For Debian info about freshness is available on https://mirror-master.debian.org, though there is no info about TLS support, for info about TLS one has to probe each server.

Though I see no info present on the site available as JSON (or other serialization format, like CBOR) and I haven't found the source of the script generating that site. @weaselp, should I implement that feature and where can I find the source?

Problem: While working on setting up an Ubuntu 18.04 build server I found that apt-mirror-updater fails to create an Ubuntu 12.04 chroot using debootstrap when running on Ubuntu 18.04:

apt_mirror_updater[18440] INFO Creating Ubuntu release Precise chroot in /var/tmp/chroots/precise-amd64 ..
apt_mirror_updater[18440] INFO Ubuntu release Precise is EOL, using http://old-releases.ubuntu.com/ubuntu/.
I: Retrieving InRelease
I: Retrieving Release
I: Retrieving Release.gpg
I: Checking Release signature
E: Release signed by unknown key (key id 40976EAF437D05B5)

Impact: The last line is an error message from debootstrap after which it exits with status code 1 and the bootstrapping process aborts. Despite Ubuntu 12.04 being end-of-life this is kind of a blocker for me: I've designed apt-mirror-updater as a general purpose library that provides a nice high level abstraction by taking care of gory details like this...

Analysis: I've since figured out what's going wrong and I'm pretty sure I know how to fix it. I'm going to use this issue to log my findings in a public place because I believe this is a bug in debootstrap that should be fixed there so that every single user of debootstrap isn't forced to work around it like I'm going to!

While researching xolox/python-executor#10 I found that the following code fails:

program = AptMirrorUpdater(
    distributor_id='debian',
    distribution_codename='testing',
)
program.create_chroot('/tmp/debian-testing')

Here are the relevant error messages:

2018-10-07 15:42:55 peter-mbp apt_mirror_updater[3668] INFO Updating package lists of chroot (/tmp/debian-unstable) ..
Hit:1 http://ftp.nl.debian.org/debian unstable InRelease
Ign:2 http://ftp.nl.debian.org/debian unstable-updates InRelease
Err:3 http://ftp.nl.debian.org/debian unstable-updates Release                 
  404  Not Found [IP: 130.89.149.21 80]
Ign:4 http://security.debian.org unstable/updates InRelease                    
Err:5 http://security.debian.org unstable/updates Release                      
  404  Not Found [IP: 151.101.36.204 80]
Get:6 http://ftp.nl.debian.org/debian unstable/main Translation-en [6211 kB]
Get:7 http://ftp.nl.debian.org/debian unstable/contrib amd64 Packages [60.8 kB]
Get:8 http://ftp.nl.debian.org/debian unstable/contrib Translation-en [50.7 kB]
Get:9 http://ftp.nl.debian.org/debian unstable/non-free amd64 Packages [96.4 kB]
Get:10 http://ftp.nl.debian.org/debian unstable/non-free Translation-en [94.3 kB]
Reading package lists... Done                                        
E: The repository 'http://ftp.nl.debian.org/debian unstable-updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'http://security.debian.org unstable/updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

For now I'll just leave this here, if it bites other people I can investigate further.

If a connection to the security repository (e.g. security.ubuntu.com) fails, AptMirrorUpdater.validate_mirror fails, causing AptMirrorUpdater.release_is_eol to return true and overwriting the possibly valid mirror with old_releases_url (e.g. old-releases.ubuntu.com)

Sometimes it is useful to be able to pass additional arguments to apt-get update (e.g. to handle bad proxies). It would be useful, for code using the API, to be able to pass these to dumb_update.

its not support support ubuntu 22.04

#0 0.544 ValueError: The string 'jammy' doesn't match a known Debian or Ubuntu release!
------
failed to solve: executor failed running [/bin/sh -c apt-mirror-updater -a]: exit code: 1

For Ubuntu-based spins (such as ElementaryOS), apt-mirror-update will raise a error due to the platform release name is different from Ubuntu.

For example: Running it on ElementaryOS will raise a ValueError: The string 'hera' doesn't match a known Debian or Ubuntu release!

Allowing users to overwrite the release when running apt-mirror-updater would fix this issue, for example: apt-mirror-updater --release=bionic

Debian Jessie is a LTS supported release, and therefore is not available on archive.debian.org after the 6 June 2018 EOL date in apt-mirror-update. It will actually move on 6 June 2020 when it goes fully EOL. apt-mirror-updater currently wrongly switches the sources.list to archive.debian.org which makes Debian Jessie installs break.