inconsistency for ssh in centos/8/cloud about lxc-ci HOT 7 CLOSED

That would be a cloud-init bug I think. We don't do anything special to our other images for that and the fact that cloud-init ran indicates that our side of things was done properly.

@raharper

from lxc-ci.

We don't like installing packages that aren't necessary for most users, especially if that causes network services to startup. I think it's best to install sshd through cloud-init if you want it.

from lxc-ci.

% lxc launch images:centos/8/cloud c8cloud 
Creating c8cloud
Starting c8cloud                            
(neipa) ~ % lxc exec c8cloud bash
[root@c8cloud ~]# cloud-init status
status: error
[root@c8cloud ~]# cloud-init status --long
status: error
time: Sun, 08 Mar 2020 19:00:25 +0000
detail:
('update_hostname', ProcessExecutionError("Unexpected error while running command.\nCommand: ['hostname']\nExit code: -\nReason: [Errno 2] No such file or directory: b'hostname': b'hostname'\nStdout: -\nStderr: -",))
('mounts', FileNotFoundError(2, 'No such file or directory'))
('set-passwords', ProcessExecutionError("Unexpected error while running command.\nCommand: ['service', 'sshd', 'restart']\nExit code: 5\nReason: -\nStdout: \nStderr: Redirecting to /bin/systemctl restart sshd.service\n        Failed to restart sshd.service: Unit sshd.service not found.",))

Cloud-init expects certain commands to be present in the image; Not having hostname or sshd service is unexpected.

IMO I believe most users of a cloud image would be surprised that they cannot ssh into it nor set the hostname. For the centos/8 image, sure, it's not a "cloud" image; for centos/8/cloud, I too would expect if you've installed cloud-init in it, for the standard boot-up to produce a happy cloud-init status.

Looking at the rpm package; we don't include a direct dependency on 'hostname', the centos8 image does have hostnamectl, cloud-init could look for hostnamectl in addition to the hostname binary. The rpm packaging could also include the sshd dependency.

@mazerty I've filed an upstream cloud-init bug on the hostname issue:

https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1866562

For the sshd requirement,

https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1866563

I suggest following up with centos8 downstream to watch these bugs and to cherry pick the fixes.

@stgraber I assume that if the downstream centos8 cloud-init rpm includes the ssh dependency than centos8/cloud image will include sshd automatically?

from lxc-ci.

Yeah, it would. Does cloudinit perform package installation first?

It would be nice if someone could just list sshd in the package list and have it be installed ahead of configuring it.

A little while back following a number of security issues, we went and removed sshd and any network service from all our images, either just not installing the package or disabling the service otherwise.

from lxc-ci.

Yeah, it would.

OK

Does cloudinit perform package installation first?

No, it must wait to install packages until the very end as installing packages may trigger reloads and spawns of new services.

It would be nice if someone could just list sshd in the package list and have it be installed ahead of configuring it.

I think this is what I'm suggesting, that if you're dnf/rpm/yum/apt installing cloud-init, that the package dependencies would also pull in sshd as one of cloud-init's primary tasks is to initialize ssh hostkeys and import user keys. It's just quite strange for a cloud-image itself to not already have sshd in it since that's by far the most typical way of accessing an instance.

A little while back following a number of security issues, we went and removed sshd and any network service from all our images, either just not installing the package or disabling the service otherwise.

I certain understand the maintenance burden.

from lxc-ci.

Hey folks, I came across something that looks closely related to this issue, but in a ubuntu/(bionic|focal|impish)/cloud images. First thing I noticed was the lack of the ssh-import package, so, I can't import any user keys to the vm/container. Then, I'm getting the ssh error:

root@automaas-test-03:~# cloud-init status --long
status: error
time: Thu, 19 May 2022 18:10:38 +0000
detail:
('set-passwords', ProcessExecutionError("Unexpected error while running command.\nCommand: ['systemctl', 'restart', 'ssh']\nExit code: 5\nReason: -\nStdout: \nStderr: Failed to restart ssh.service: Unit ssh.service not found.",))

But what is curious is that after the VM is up I can restart the ssh command. Also, after the VM is started, I can't ssh to the VM
due to the profile being locked but SSH does work.

Is there something missing from the config Im using or is this a bug in cloud-init or lxd?

My profile is this: https://gist.github.com/sombrafam/f85ed367ff6eabf8741fe2d47dd125bb

from lxc-ci.

Just figured it out. I'm adding the ss-import package in the list which install the openssh-server as dependency. But, this still kind of a problem. What is the solution for this case? cloud-init still blocking the user from sshing into the instance since the ssh_authorized_keys are inserted but the user is locked due the cloud-init module breaking.

from lxc-ci.