xenserver / host-installer Goto Github PK

If an unpacked driver-disk ISO is used through a <driver-source type="url"> in answerfile, and the isRepo() check fails because the directories are not readable, the driver source is silently ignored: no error shown to user, and no diagnostic written to install-log.

Other error cases are likely causing a repo to get silently ignored, we should be more robust.

Found this issue by unpacking the ISO using 7z tool, which unfortunately otherwise seems to be the most practical tool to extract files from an ISO image (with mount requiring root permissions, fuseiso not being universally available, and still needing more manual operations than 7z (correctly) teaches users to expect, and similarly for the more recent udisksctl loop-setup).

When making an install with v10.10.8 and selecting all default choices in the TUI, during installation the user is presented with a 'NoneType' object is not iterable error and can only reboot.

The log shows it happens in rewriteNTPConf while attempting to iterate on ntp_servers.

Cannot guess the source while just looking at the NTP rework patch, and cannot bisect it either: that big commit doing several things demonstrates the value of splitting into individual commits (eg. "Replace time-config-method with ntp-config-method to prepare for DHCP support", "Separate NTP servers manual selection into a separate dialog", "Introduce option to get NTP servers from DHCP", and likely more that whose need would naturally arise as separate from those 3 main topics during careful review if the patches).

Likely linked with #64.

Hi!

So, this is a question more about XenServer installation images than about host-installer, but this is the closest bugtracker I can think of to ask it.

So I know UEFI boot for CDs is kind of complicated, and that's why there's some duplication of grub and its configuration on the ISO.

Now, there are files we really can't find a use for the hardest we think, so I wondered if you could explain them to us, or maybe have a fresh look and maybe just remove them if they don't have any purpose anymore.

1. /EFI/xenserver/grub-usb.cfg: The name suggests it's useful to boot from USB, but we can't find any reference to it in grub code, nor any code that would expand on the grub.cfg name to add a suffix to the first part. I don't find it referenced in XenServer documentation either. The same file is included in efiboot.img under the name grub.cfg.

2. /boot/gcdx64.efi. This is a build of grub with a different prefix for the grub configuration (/EFI/BOOT instead of /EFI/xenserver), but we see no way that this binary is loaded at boot time. A similar question can be asked about /boot/grubx64.efi, which is a copy of the other two instances of grub in the ISO (one in efiboot.img, one in /EFI/xenserver/). I don't find it referenced at that exact /boot/grubx64.efi path in XenServer documentation, so I doubt it's useful, even for PXE boot.

Are we missing something in the complicated world of UEFI CD/USB/HDD ISO booting?

CC @ydirson who will be interested too in any insight.

There is a big discrepancy in handling of local driver disks by the TUI: any driver disk of type "local" that was installed in the installer environment does not by default get installed on the host, whereas any of type "url" will.

Possible options include:

1. notifying the user that they will need to add them using the "supplemental packs" dialog too
2. explicitly request the user to insert each driver disks at the end of main product installation
3. cache the driver disk contents in the installer's ramdisk environment, as they are typically quite small (and a caching limit could be used, falling back to 1. or 2. if needed)

While working on LACP bonding support, I realize that VLAN configuration on cmdline allows some non-sensical configurations like:

--network_device=all --network_config=dhcp:vlan=10

Also "just specifying a VLAN on a given interface" requires today to repeat the dhcp default with no good reason:

--network_device=eth0 --network_config=dhcp:vlan=10

I think we should move the VLAN specification to the --network_device option, like

--network_device=eth[:vlan=VLAN]|mac[:vlan=VLAN]|all

The "just specifying a VLAN on a given interface" usecase will then become:

--network_device=eth0:vlan=10

When host has a single disk, the "Select Primary Disk" screen is skipped. In this case, hitting "Back" from "Virtual Machine Storage" screen gets back to "Select Primary Disk" which is not shown, and the user is just presented "Virtual Machine Storage" again: "Back" simply does not work.

Changing the "Select Primary Disk" to be displayed even with a single disk would not only solve this issue, but would also let the user have a better understanding of what happens, as

• that screen is the one saying "disks with insufficient space are not shown"
• some users reportedly confuse the "Virtual Machine Storage" screen with primary disk selection when there is no screen shown for the latter

Currently, udev rules in /lib/udev/rules.d/65-md-incremental.rules cause any pre-existing software-RAID devices to be auto-assembled, and the user cannot opt-out. As a consequence:

• disks that are part of a RAID array of disks cannot be selected for install
• a RAID array built from partitions can be selected for install (but installed system won't boot)
• disks with partitions that are part of a RAID array are available for selection, but install will fail

To solve this situations, our current plans for XCP-ng are:

• disable auto-assembly (eg. by causing the ANACONDA udev envvar to be set), solving all the above problems
• still allow user to enable RAID when needed, identified use cases including:
  • being able to upgrade or restore a system that was installed in RAID device (we already have support for such installs, not contributed yet, but I understand NetScaler has a use-case)
  • being able to install on a pre-existing RAID (would be the NetScaler use-case)

The basic idea for now is to probe the available disks for RAID superblocks (mdadm --examine), and when one is found, add an option besides the existing install/upgrade/restore/reinstall ones, to assemble any RAID volumes (mdadm --assemble --scan). An option for this would be provided to activate this behavior from answerfile.

This will notably require moving the "Checking for existing products" logic later, so it can be rerun. Its current location, before ensuring there are any disks in the system, already feels awkward; is there anything to be aware of around that area ?

Digging into initial SR creation, I find a couple of inconsistencies, not all of which are obvious to get straight:

• default-storage.conf gets written both XSPARTITIONS/XSTYPE and PARTITIONS/TYPE, with backend.py saying the latter are legacy names. It would appear that scripts on a newly installed system will use non-legacy names and have no need for legacy ones, so probably we could remove legacy names. However storage-init does use PARTITIONS/TYPE. Something looks wrong :)
• backend.py points to prepare-storage as user of this file, while the user is clearly storage-init

@rosslagerwall, those XS names were introduced in 2008, is this some transition that did not make it into sm ?

Currently, when the system is installed using, say, legacy boot, and a user subsequently changes to UEFI for booting a new install media, the choice to upgrade the installed system is still offered, even though it will not work, and will possibly not be detected until late post-installation phase.
Similarly, and possibly more likely, if a backup exists, which can date from before a voluntary switch from legacy to UEFI, it will be offered with similar consequences.

It should be doable to record the boot-type in /etc/xensource-inventory so we can avoid early such situations. Another option (or fallback option) could be to rely on the /etc/fstab contents to check for an /boot/efi entry.

Does it look like something that could be accepted ?

Adding the same note than after install would make the restore process more consistent with the rest. After a terse "The restore operation completed successfully.", the reboot immediately on clicking OK can come as a surprise.

When using the advanced boot command shell, I get the following:

This does not include any option added by the preinit script, and by default the --reboot flag is added.

When using the network to access driver repositories and main repository, the TUI allows to fill the "Networking" form only once, and the reuse it through "Use existing configuration".
Then when it comes to the host network network config, we directly get the "Networking" form, which in many cases will be filled with the same parameters used for the install. It would be useful to be able to select "Use existing configuration".

My recently-merged "tui: show the disk on which an existing installation or backup is found" patch reveals an old issue.

In a nested BIOS install a backup detected as <XenServerBackup: XCP-ng 8.3 (8.3.0-cloud) on /dev/sda2>, where the PRIMARY_DISK is /dev/disk/by-id/nvme-QEMU_NVMe_Ctrl_nvme0 (denoting that backup was originally installed as UEFI), is presented to the user as a valid choice to restore from.

However, since 10.10.9 we now display more detailed entries, and this causes a exception, as:

• getDiskDeviceSize("/dev/disk/by-id/nvme-QEMU_NVMe_Ctrl_nvme0") returns None because of a missing else: branch (dev = "disk!by-id!nvme-QEMU_NVMe_Ctrl_nvme0" founds nothing matching /sys/block/%s/device/block/size or /sys/block/%s/size)
• getExtendedDiskInfo() attempts to divide None / 2048, which causes the installer to abort

No such problem when booting the installer from UEFI, the backup entry succeeds in getting formatted.

The disk!by-id!nvme-QEMU_NVMe_Ctrl_nvme0 string, where in UEFI boot /sys/block/ contains nvme0n1 makes me suspecting we're trying to find a non-existent device, do not catch the failure to get its realpath, and go on using the bad device path.

This issue is linked to #11, in that a UEFI backup should be considered invalid on a BIOS install, and not proposed.
Note that in theory any PRIMARY_DISK pointing to a "missing disk" would lead to this, this could e.g. also happen if in a future platform udev rules change the /disk/by-id/ link.

v10.10.7 changes the way NTP is handled to cope with new support for getting NTP servers from DHCP. This is real good news.

However, the new source="default" behavior seems strange in several ways:

• the documentation does not document it (what does "use default NTP servers" as alternative to well-defined dhcp|manual|none choices?) Likely the "the default shall be..." sentence is meant to explain that, except that it does not explicitly states it applies to default, which is likely not explicitly documented as being the default value if the ntp element is not provided.
• "Use default NTP servers" in the TUI is likely to cause confusion; what would be the reason not to select an actual source? Its effect is even not clear at all from the code.
• the default-selection log exists in answerfile.py but leaves a branch where results['ntp-config-method'] = "default", which should likely not be

A Driver Disk generated with the DDK but with an error in groups.xml referencing a non-existent but mandatory package (<packagereq type="mandatory">update-</packagereq>) results in this during install:

     YUM: nothing to do
     YUM stderr: Warning: Group drivers does not have any packages to install.

... then the installer UI still reports the driver disk as "loaded", even though the package containing said driver has not been installed.

I realize this may originate in a problem with yum disregarding the "mandatory" type, but I'm stll quite new to package groups.

parseInterface() code is as follows:

        if_name = getStrAttribute(node, ['name'])
        if if_name and if_name in nethw:
            if_hwaddr = nethw[if_name].hwaddr
        else:
            if_hwaddr = getStrAttribute(node, ['hwaddr'])
            if if_hwaddr:
                matching_list = filter(lambda x: x.hwaddr == if_hwaddr.lower(), nethw.values())
                if len(matching_list) == 1:
                    if_name = matching_list[0].name
        if not if_name and not if_hwaddr:
             raise AnswerfileException("<admin-interface> tag must have one of 'name' or 'hwaddr'")

It is allowed to specify both name and hwaddr, in which case

• if name points to an existing interface, hwaddr is ignored
• if not, name is ignored

If any or both are specified, but do not match the hardware, the message says they are missing.

We should likely ensure exactly one is given, and issue more specific error messages.

IIUC DebStyleInterface related code is dead and should be removed since now its the RH style that is used?
Am I correct in my assumption? Would you be okay if I open a PR to delete this code?

Would it be possible to get a v10.10.9 installer tagged, so we get an official base version, quite some of our PRs were merged post-10.10.8. TIA!